Updated 2018-01-07 11:30:06 by pooryorick ▲ (Redirected from closure)

In other programming languages, a closure captures the environment in which a routine is defined, allowing the routine to refer to symbols that exist in that environment.

See Also  edit

A brief introduction to closures and continuations for event-based programming
The title says it all.
Emulating Closures in Tcl
Simple Closures and Objects
Playing Scheme
sproc
AMG: I'm pretty sure it provides closures.
stowroutine, by PYK
Procedures that get their own namespaces to play in.
a collection of Tcl utilities, by The Computer Architecture Group at the University of Heidelberg
Includes a closure package.

Further Reading  edit

Wikipedia
C2
Perl

Description  edit

Although closures are closely associated with lexical scoping, that's more of an artifact of the languages that currently implement them than of some intrinsic relationship between the two things.

A Tcl Chatroom Discussion  edit

dkf
Aliases are sort-of poor-mans closures. If we had real closures, we'd use them.
suchenwi
What prevents us from having them?
dkf
Lifespan issues mainly.
am
Why are aliases not closures - sorry, I think I have asked this before, but I do not fully grasp the concept of a closure
dkf
If we could be sure that the closure would go away as soon as it was no longer needed, they'd be trivial to add. A closure is really a piece of stack frame that is not on the stack. Or something like that. (The really strange things are continuations, but we're not dealing with those now.) Now it would be pretty easy to create a new closure object and commands to manipulate that, but ensuring that it doesn't go away at the wrong moment is not entirely trivial.
suchenwi
Could static variables be called atomar closures? I meant in the sense that they're local to a function, but persistent between calls.
dkf
C's static variables? Sort of. Not that they're exactly that. Closures are really a bit antithetical to C anyway.
dkf
However, I could envision a closure/stack-frame object that you'd access through upvar A proper closure keeps track of all enclosing stack frames, but that's probably more than is needed for most purposes. Then you could also store the object as an argument to the proc using aliases...

And that'd work. So long as the closure doesn't get disposed of too early. We really need to extend Tcl_ObjType when we come to [Tcl9].

Lars H, 2010-02-05: It seems Apple is bent on adding some kind of closures to C, though [1].

Misc  edit

glennj: It took me a while to figure out how aliases could work like closures in the above discussion. I think I came up with an example: a procedure returns the number of times it's been invoked:
# the first time, send "0" to the aliased proc
interp alias {} countMe {} _countMe 0

proc _countMe {count} {
    # for the next invocation, reset the alias
    interp alias {} countMe {} _countMe [incr count]
    return $count
}

puts [countMe] ;# ==> 1
puts [countMe] ;# ==> 2
puts [countMe] ;# ==> 3

Todd Coram:

While playing with PostgreSQL, I got tired of passing around the connection id to each postgres tcl command (plus there was some other contextual info I wanted to keep around with each connection).

Objectifying the postgres interface seemed over-kill and keeping state in a single namespace prevented me from keeping more than one database connection open at a time. I could have used the classic objectifying technique of keeping the instance info in array, but this felt clumsy. I could have curried the connection id to postgres commands, but that felt too restricting.

Closures are neat because they give you a way to let functions carry a bit of state with it. Tcl Namespaces give you most of what you need, but you start to bleed into object land when you try to create more than one exclusive set of variables.

So, thus was born this late night hacked attempt at emulating closures in Tcl:
# Create a proc named 'name' that will create a closure over the supplied
# 'variables' definition for the 'lambda_proc'. Any supplied variables in
# 'arglist' can be used to initialize 'variables' during the closure
# definition.
#
proc make-closure-proc {name arglist variables lambda_proc} {
    set invoke_context [uplevel namespace current]
    set name_context ${invoke_context}::$name

    # Create a namespace called $name_context to hold auto_cnt
    #
    namespace eval $name_context {
       if {![info exists auto_cnt]} { variable auto_cnt -1}
    }

    # Now, build a proc in invocation context that will create
    # closures. We do this by substituting all of the passed
    # parameters (name, arglist, variables, lambda_proc) and the
    # $name_context.
    #
    # The resulting proc will:
    # 1. Accept $arglist as initializers for the closures.
    # 2. Create a unique closure_name from the auto_cnt variable.
    # 3. Create a namespace for the closure.
    # 4. Evaluate the $variables (optionally evaluating them with
    #    $arglist).
    # 5. Create an alias called 'dispatch' for the lambda_proc.
    # 6. Return the alias.
    #
    namespace eval $invoke_context \
       [subst -nocommands -nobackslashes {
           proc $name {$arglist} { 
               set closure_name \
               ${name_context}::$name[incr ${name_context}::auto_cnt]
               eval [subst {
                   namespace eval [set closure_name] {
                       $variables
                   }
               }]
               namespace eval [set closure_name] {
                   # Curry a dispatcher for the lambda_proc.
                   #
                   curry [namespace current]::dispatch [$lambda_proc]
               } 
               return [set closure_name]::dispatch}
            }]
}

proc delete-closure {name} {
    namespace delete [namespace qualifiers $name]
}

I used a curry proc to create the dispatcher (I really don't create a curry and perhaps namespace code could accomplish a similiar thing, but I had a curry proc handy, so there):
proc curry {new args} {
    uplevel [list interp alias {} $new {}] $args
}

And a lambda proc to pass to the make-closure-proc:
proc lambda {arglst body} {
    set level [info level 0]
    set name [string map {\n _ \t _ \" _ " " _ \; _ $ _ : _ \{ _ \} _ \[ _ \] _} $level]
    set invoke_context [uplevel namespace current]
    proc ${invoke_context}::$name $arglst $body
    return ${invoke_context}::$name
}

Here is a (contrived) example of how to create and use the closures:
make-closure-proc make-logputs {_level _filename} {
    variable fd [open $_filename w]
    variable filename $_filename
    variable level $_level
} {
    lambda {cmd {str ""}} {
        variable fd; variable filename; variable level
        switch -- $cmd {
            puts { 
                puts "Writing ($level to $filename) $str"
                puts $fd "([clock format [clock seconds]] - $level) $str"
            }
            close { 
               close $fd 
            }
        }
    }
}

set info [make-logputs INFO info.out]
set warn [make-logputs WARN warn.out]

$info puts "Some info: hello world"
$info puts "blech"
$warn puts "You have been warned!"

$info close
$warn close

delete-closure $info
delete-closure $warn

GN There is a strong relation between objects and closures. a classical paper is from Uday Reddy "Objects as closures: abstract semantics of object-oriented languages", who defines objects as an "message environment" (binding messages to methods) with an hidden local environment (binding instance variables to values).

This is pretty close to the notion of objects in XOTcl; the example above can be written much simpler with XOTcl (certainly with other oo languages as well):
package require XOTcl; namespace import -force xotcl::*

Class Logger -parameter {level filename}
Logger instproc init {} {
    my instvar fd filename
    set fd [open $filename w]
}
Logger instproc puts {str} {
    my instvar level filename fd
    puts "Writing ($level to $filename) $str"
    puts $fd "([clock format [clock seconds]] - $level) $str"
}
Logger instproc close {} {
    my instvar fd
    close $fd
}

set info [Logger new -level INFO -filename info.out]
set warn [Logger new -level WARN -filename warn.out]

$info puts "Some info: hello world"
$info puts "blech"
$warn puts "You have been warned!"

$info close
$warn close

$info destroy
$warn destroy

TV Isn't closure like in physics where you'd have a mathematical construct which is 'closed', which is usually pretty darn hard and often quite useless except probably in some essential fundamental cases.

DKF: That's essentially unrelated. (And that's a maths/topology concept which physics has borrowed.)

See custom curry and the references there for an explanation of curry and currying function.

RS asks: Could one say the following are (rudimentary) examples of closures?

1. Default arguments (in danger to be overwritten by caller);
proc foo {a b {c 1} {d 2} {e 3}} {...}

2. Name-value map in an explicit argument
set closure [list c 1 d 2 e 3]
...
proc bar {a b closure} {
    foreach {name value} $closure {set $name $value}
    ...
}

3. Closure values passed in over an alias:
proc _grill {c d e a b} {...}
...
interp alias {} grill {} _grill 1 2 3

Zarutians first attempt at closures:
package require Tcl 8.4
proc closure_var args {
    set varname [lindex $args 0]
    set value   [lindex $args 1]
    upvar $varname breyta
    set breyta $value
    trace remove variable breyta {write} update_closure_var ; # to prevent multiple trace clones
    trace add variable breyta {write} update_closure_var
}
proc update_closure_var {varname1 varname2 op} {
    if {$varname2 != ""} {
        set varname "[set varname1]([set varname2])"
    } else {
        set varname $varname1
    }
    upvar $varname newvalue
    debug [info level 1] $varname $op $newvalue
    # redefine the calling proc to reflect the variables new value

    set procsName [lindex [info level 1] 0]
    set procsArgs [info args $procsName]
    set procsBody [info body $procsName]
 
    debug $procsName $procsArgs
    debug $procsBody
    
    set temp "closure_var $varname"
    set insertLoc [expr [string last $temp $procsBody] + [string length $temp]]
    set firstHalf [string range $procsBody 0 $insertLoc]
    set lastHalf  [string range $procsBody $insertLoc end]
    set procsBody "[set firstHalf]\{[set newvalue]\}[set lastHalf]"
    
    debug $procsBody
    
    proc $procsName $procsArgs $procsBody
}
proc debug args { puts "$args" }
proc someFunction {} {
    closure_var blu 21
    set blu 234
}

The above implemention doesn't handle that when other sub-procs upvar a closure_var . The next implemention of closures from Zarutian will have that fixed.

Zarutians second attempt at closures
package require Tcl 8.4

proc closure_var args {
    set varname [lindex $args 0]
    set value   [lindex $args 1]
    upvar $varname breyta
    set breyta $value
    trace remove variable breyta {write} [list update_closure_var $varname]; # to prevent multiple trace clones
    trace add variable breyta {write} [list update_closure_var $varname]
}
proc update_closure_var {originalVarname varname1 varname2 op} {
    if {$varname2 != ""} {
        set varname "[set varname1]([set varname2])"
    } else {
        set varname $varname1
    }
    upvar 1 $varname newvalue
    debug [info level 1] $varname $op $newvalue
    # redefine the calling proc to reflect the variables new value

    set procsName [lindex [info level 1] 0]
    set procsArgs [info args $procsName]
    set procsBody [info body $procsName]

    debug $procsName $procsArgs
    debug $procsBody

    set temp "closure_var $originalVarname"
    debug $temp
    set insertLoc [expr [string last $temp $procsBody] + [string length $temp]]

    set firstHalf [string range $procsBody 0 $insertLoc] 
    set lastHalf  [string range $procsBody $insertLoc end]
    set procsBody "[set firstHalf]\{[set newvalue]\}[set lastHalf]"

    debug $procsBody

    proc $procsName $procsArgs $procsBody
}
proc debug args { puts "$args" }
proc someFunction {} {
    closure_var blu 21
    decr blu
}
proc decr {varname {delta 1}} {
    upvar 1 $varname var
    set var [expr $var - $delta]
}

This implemention should work fine with other sub-procs upvaring closure_vars. But these two implementions have one catch/issue: namely that every previus value of a closure_var is saved too. I am trieing find out how it is possible to get the old value of a variable before it is updated to the new value so I can fix that catch/issue

Just to be sure: isn't a closure just a procedure with some state saved with it, no?

SS 2004-12-30: For the Tcl semantic, lexical binding is hard because it's not possible to tell from the source code what's a reference to a variable and what is not. Still it's possible to capture the context of a lambda's creation saving the current stack frame, thus allowing for real closures (procedures where if some variable can't be resolved in the local context, will try to access the stack frame that was active at the procedure creation time). Because of the way Tcl works and the fact it's so dynamic I'm not sure closures may significantly help Tcl programming, but when they are used as objects. It's a more interesting addition to have an object system with garbage collection. Because in most Tcl objects systems objects are "callable", like in [$o a b c], objects can be used to do most of the things you may like to do with closures, they are very similar to procedures but can take/alter their internal state. Of course this is possible to do even with OOP systems not having garbage collection, but the programming style that closures make possible is often about to capture some information to semplify/specialize the next uses of a given procedure, it's hard to think at this programming style as comfortable if you have to clean by hand the objects.

Zarutian 2005-01-02:

Append this to my second attempt at closures:
proc closure {name args body} {
    set locals [uplevel 1 [list info locals]]
    foreach loco $locals {
        set body "[list closure_var $loco [uplevel 1 [list set $loco]]]\n[set body]"
    }
    proc $name $args $body
}

and I think you have got something like that SS described above.

SS 2005-01-03: Yeah this is a smart idea, to capture the context. Still there are a few things to fix, and many other things that we can't fix at all. What should be fixed is that this way you set the locals variables even if the name is the same as one of the procedure arguments, that's not correct: if there is an argument with that name, the symbol should be bound to it. Second, in real closures, many closures can *share* the environment, and can modify it (and every other closure bound to the same external variables will see the change). We can't get that far with Tcl of course, but still your exercise is very interesting and valuable IMHO.

DKF: I use something very much like closures in my oo2 code. Basically, dict's update and with (together with some wrapping scripts) allow for closure-like behaviour very easily. The complexity is that the closure is actually stored in a "global" variable, but as long as you ignore the man behind the curtain, you're OK... ;^)

Zarutian: As I understand it Tcl procedure's environment is three leveled: global, local to procedure's namespace, local to procedure's level/callstack.

Then when variable lookup is performed (either because of Tcl's seventh substitution rule or procedure set is invoked) inside a procedure's body (the procedure was called) the interpreter first looks for the variable in local callstack scope then in procedure's namespace (if that variable name was marked by calling the procedure 'variable') and at last in the global namespace (likewise if the variable name was marked by calling the procedure 'global').

Where other scripting languages like Lua and Lisp variants use linked tables/hashmaps/dictionaries and lookup the variable's value in the parent table of the current variable|value binding table if not found in current until the top most variable|value binding table is reached. (Hmm... the preceeding sentence may be unclear because of my lack of skill writing English)

Here above I have descriped two possible ways to look up values for variables. So I ask what are the 'pros' and 'cons' of each?

In my opinion I think the latter method enables more flexibility of variable lookup for value. {IS: Að mínu áliti held ég að síðari aðferðin gefi kost á meiri liðleika til að fletta upp gildi breytu.}

SS 2005-02-23:

This is a proposal for closures I made in comp.lang.tcl:

From time to time there is this discussion about adding closures to Tcl, but usually the idea is to make they similar to languages like Scheme, i.e. closures with lexical scoping. My opinion is that this is very hard to do with Tcl, and even not in the spirit of the language. In a Tcl program, what part of a procedure body is a variable is not defined until execution, for example:
set a 10
closure {x} {
    incr a
}

Then I define incr to be like puts, and a is no longer a variable. It was already suggested that in Tcl closures should not have any kind of creation-time resolution rule, but that the context where they are defined should be captured as a whole, and used when the closure is running to resolve "unbound symbols" (that are better referred as variables and procedures not otherwise defined during execution in Tcl slang). Still there is the problem that to similuate the lexical scoping you have to take *references* to the shared environment, so that:
set a 10
set foo [closure {x} {
    incr a $x
}]
set bar [closure {x} {
    incr a $x
}]
$foo 1;# -> 11
$bar 1;# -> 12

foo and bar will share the same 'a'.

I think this is a all to complex and UnTclish, so I tried to design a new semantic for closures.

The semantic is based on a single command, closure, that can set/get closure variables in the scope of the current procedure, together with a minimal change in proc and lambda (I know we don't have lambda... but still there is a way Tcl users already think about it and should change).

Basically, to set a closure variable, there is to write the following command:
closure set x 10

This will set $x inside the closure of the current procedure. This $x will be persistent accross different calls of this procedure, and all this environment will be destroied once the procedure itself is destroied. The procedure can test for the existence of a closure variable with [closure exists x], can get the value with [closure get x] and so on.

The change required to the proc is that it can take an optional further argument, a Tcl list of key/value pairs used to inizialize the closure at procedure creation time.

so:
proc foo {} { .... } {x 10 y 20}

will create the procedure foo with x=10 and y=20 inside the closure. The same for lambda.

The following is an example of procedure that returns a progressive integer number every time it's called:
proc counter {} {
    if {![closure exists x]} {
         set x 0
    }
    set x [closure get x]
    closure set x [expr $x+1]
    return $x
}

Of course it's better to use the optional argument to proc and write it as:
proc counter {} {
    set x [closure get x]
    closure set x [expr $x+1]
    return $x
} {x 0}

Of course for this to be very useful, lambda it's needed. This version of lambda, like proc, should accept the optional argument to initialize the closure. This is an example:
proc createAdder {x} {
    lamba {y} {expr $y+[closure get $x]} [list x $x]
}
 
set f [createAdder 5]
$f 10 ;# -> 15

I think that this design can do everything lexical scoping is able to do, but with a command-based interface that plays better with Tcl.

This kind of closures will be added into the Jim interpreter, a small footprint Tcl interpreter I'm writing. This interpreter is already working and I'll be happy to send a preview tar.gz to interested people (the license is the APACHE2, so it's possible to use Jim in commercial projects if needed).

Regards,
Salvatore

and then I replyed to myself with this message:

I forgot to include a couple of points.

The closure command can also access closures of external procedures:
 [closure in $procName get/set x ...] and so on.

This makes possible to share the environment if really needed. Example:
proc createAdder {x} {
    set a [lamba {y} {expr $y+[closure get $x]} [list x $x]]
    set b [lamba {incr} {closure in $a set x $incr} [list a $a]]
    list $a $b
}

This returns one closure to add and one to change the increment performed by the first closure.

Also it is worth to note that this way to do closures don't play well with functions as values nor with lambas based on auto expansion, because full closures need to be able to modify their environment so they can not be immutable values.

Ciao,
Salvatore

NEM offers the following as a simple closure-like thing:
proc closure {name arglist body} {
    if {![string match ::* $name]} {
        # Not fully qualified
        set ns [uplevel 1 namespace current]
        set name ${ns}::$name
    } else {
        set name $name
    }
    set sname [namespace tail $name]
    # Just create a proc as usual, but a namespace along with it
    namespace eval $name [list proc $sname $arglist $body]
    interp alias {} $name {} ${name}::$sname
}

and a test:
closure make-counter {start} {
    variable id
    if {![info exists id]} { set id 0 }
    incr id
    closure counter-$id {{amount 1}} [string map [list %S $start] {
        variable counter
        if {![info exists counter]} { set counter %S }
        incr counter $amount
    }]
    return [namespace current]::counter-$id
}

Could do with a bit of tarting up, and it's not as powerful as closures in Scheme, which get a lot of their power from lexical scoping. Still, it goes quite a long way, I think. Oh, and you can do drop-dead simple ensembles too:
(tile) 57 % closure MyEnsemble {method args} {

  closure say {message} { puts "Say: $message" }
  closure sing {message} { puts "Sing: $message!" }

  $method {*}$args
}
::::MyEnsemble
(tile) 58 % MyEnsemble
wrong # args: should be "::::MyEnsemble::MyEnsemble method ..."
(tile) 59 % MyEnsemble say Hello
Say: Hello
(tile) 60 % MyEnsemble sing "Tra-la-la"
Sing: Tra-la-la!

Sarnold 2010-02-05: Here is my own implementation of simple closures. To use them you will have to store them in a variable and pass its name to a custom applyc (apply-closure) proc, which saves the environment in the variable holding the closure. The closure proc creates a closure (a list) by passing to it a variable list for the environment. (this is AFAIK the way you create closures in PHP)
package require Tcl 8.5

proc closure {arglist args} {
    # returns a list of 4 elements:
    # - the argument list of apply
    # - the closure var list
    # - the closure vars initial values (to be updated by applyc)
    # - the body
    switch -- [llength $args] {
        1 {return [list $arglist "" "" [lindex $args 0]]}
        2 {lassign $args varlist body}
        default {error "bad arguments to proc closure"}
    }
    if {[llength $varlist]==0} {return [list $arglist "" "" $body]}
    foreach var $varlist {
        lappend data [uplevel 1 set $var]
        set body "upvar 1 closure_$var $var\n$body"
    }
    list $arglist $varlist $data $body
}

proc applyc {name args} {
    upvar 1 $name _closure
    lassign $_closure arglist varlist data body
    foreach var $varlist val $data {
        set closure_$var $val
    }
    # call the lambda
    set res [apply [list $arglist $body] {*}$args]
    # variable updates are saved in local vars
    foreach var $varlist {
        # retrieve them
        lappend values [set closure_$var]
    }
    # save them as third element of _closure
    lset _closure 2 $values
    set res
}


set x 1
set accu [closure {num} {x} {incr x $num}]
puts $accu
puts [applyc accu 2]
puts [applyc accu 3];#should increase value
proc lam {lam} {applyc lam 2}
puts [lam $accu]
puts [lam $accu];#should print the same

NEM 2010-02-06: This is essentially how dictutils works, except that in that implementation the lambda expression and the variable environment (a dict) are separate arguments to the apply command. Both have the nice property that updates are only committed to the environment if no error is thrown in the body of the lambda. Both however need a little work to ensure that the lambda expression can be byte-compiled properly.

Thinking some more, a minimal closure construct that does the job and should be properly byte-compiled (but still not entirely optimised) is the following:
proc closure {env params body} {
    set ns [uplevel 1 { namespace current }]
    set body [list with __scope__ $body]
    set lambda [list $params $body $ns]
    return [list $env $lambda]
}
proc with {scopeVar body} {
    uplevel 1 [list upvar 1 $scopeVar __scope__]
    uplevel 1 [list dict with __scope__ $body]
}
proc applyc {closureVar args} {
    upvar 1 $closureVar closure
    lassign $closure __scope__ lambda
    try { apply $lambda {*}$args } on ok result { set closure [list $__scope__ $lambda]; return $result }
}
# EXAMPLE
set counter [closure {i 0} {{n 1}} { incr i $n }]
applyc counter ;# -> 1
applyc counter ;# -> 2
applyc counter 4 ;# -> 6 etc

Category ConceptCategory ExampleCategory XOTcl Code