package in Tcllib.Richard Suchenwirth - Spent some time this weekend looking at other languages (Python, Lisp):- what do they have which Tcl hasn't? What feature or functionality is missing in Tcl?
- is it worth having?
- how can we do it in Tcl?
set L {10 20 30 40 50}
set res {}
foreach i $L {lappend res [expr $i*1.16]}
set L2 $resNow if you want to wrap the single-element processing and the list processing in separate procs, you might say: proc addVAT {x} {expr $x*1.16}
proc map {fun list} {
set res {}
foreach i $list {lappend res [$fun $i]}
return $res
}
set L2 [map addVAT $L]Now if you don't want to invent a proc name for each single-element proc, you're ripe for Lambda: given an arglist and a body, add a name, and make it a proc. One proposal: lambdas are anonymous - no name - empty string (which is a valid proc name in Tcl), so: proc lambda {argl body} {
set name {}
proc $name $argl $body
return $name
}
set L2 [map [lambda x {expr $x*1.16}] $L]This implementation has the feature that always the same name is used: a kind of garbage collection, but nested lambdas wouldn't work. If you want nonconflicting names for the lambda object, you might use "very telling names": use the lineset name $argl/$bodywhich completely describes the function being defined, and will not leak if the same lambda is called more often. Luckily, the body is often very short in lambdas...;-)On the other hand, in order to define a "proc with no name", or rather the empty string as name, we need not even define a lambda proc as above. Tcl's proc command returns nothing, which is the empty string, which happens to be the return value of lambda in the first alternative. So we may say
set L2 [map [proc "" x {expr $x*1.16}] $L]Now, if anyone slightens Tcl for not having a Lambda function, tell them it's called "proc {}" and was there all the time (just we didn't think of it...;-) You can make this lightweight lambda look nicer with interp alias {} lambda {} proc {}
set L2 [map [lambda x {expr $x*1.16}] $L]And then: Of course we can model all the functional programming goodies in Python (see also Steps towards functional programming), but there may be cooler ways: To add up the numbers in a list, you may
set sum [reduce [lambda {p q} {expr $p+$q}] 0 $L](that's how it's done in Python, translated to Tcl, but:)set sum [expr [join $L +]+0]does the same job and looks definitely slicker... (but see Elegance vs. performance for how slow it is) -- Richard Suchenwirth This was the init of a lengthy discussion on news:comp.lang.tcl
briefly summarized here: Larry Virden has Lambda-FAQ: http://www.purl.org/NET/Tcl-FAQ/part5.htmlDonal Fellows presented a lambda namespace that never forgets ;-) namespace eval lambda {variable unique 0}
proc lambda {argList body} {
set name ::lambda::cmd[incr ::lambda::unique]
proc $name $argList $body
return $name
}and reported that reducing with join is much faster: % time {set sum [reduce sum 0 $L]} 10000
781 microseconds per iteration
% time {set sum [reduce [lambda {p q} {expr $p+$q}] 0 $L]} 10000
867 microseconds per iteration
% time {set sum [eval expr [join [concat 0 $L] +]]} 10000
489 microseconds per iterationPaul Duffin has lambda, and curry, in his Feather extension.[China Blue] offers lambda, and map, without using proc inside: proc setp {vars vals} {
set m [llength $vars]
set n [llength $vals]
while {$m>$n} {lappend vals {}; incr n}
uplevel 1 [list foreach $vars $vals break]
lrange $vals $m end
}
proc lambda {locals body args} {
set args [setp $locals $args]
eval $body
}
proc map {func list} {
set r {}
foreach e $list {lappend r [eval $func $e]}
return $r
}Later, some diversions of Common Lisp readmacros for Tcl, or a Tcl readmacro for Lisp. DejaNews has the details -- RS An in-depth document on Tcl lambda is at http://homepages.ihug.co.nz/~webscool/itp/Lambda/Lambda.htmlwdb In Lisp (esp. Scheme), lambda is integrated to the concept of the language. An anonymous procedure such as (lambda (x) (list 'x x)) does exactly the same as the value of symbol name-it resulting from the function (define (name-it x) (list 'x x)).In Tcl, the procedure proc name-it x {list x $x} does perhaps the same, but hides that it knows of its containing namespace where known external procedures reside as well as variables which can be integrated via variable.In my private projects, I have self-made lambdas, of course. They are regular procedures, residing in their own directory. But they differ from "natural" procedures in such that they neither "see" the procedures nor variables in the namespace where they are called. Result are surprising bugs.Add it (it's easy), enjoy it. But do not add it to the core.
The life of a lambda proc can also be shortened with the method presented in Local procedures, where a proc is unset on the unsetting of a variable in the caller scope:
proc lambda {args body} {
set name lambda[intgen]
uplevel 1 set __$name -
uplevel 1 "{trace variable __$name u {rename $name {} ;#}}"
proc $name $args $body
set name
}
with the old faithful numbering machine ''intgen'' making unique IDs:
proc intgen {{seed 0}} {
set self [lindex [info level 0] 0]
proc $self "{seed [incr seed]}" [info body $self]
set seed
} ;# RSStrangely enough, lambda can also be used to model data structures! Tcl and Lisp shows one implementation of lambda used to do Lisp's lists, with absolutely no data structure other than the lambda!
Re-thinking on Lambdas, I like the following best:
proc lambda {args body} {
set name [info level 0]
proc $name $args $body
set name
}
% set inv [lambda x {expr 1./$x}]
lambda x {expr 1./$x}
% $inv 5
0.2The name of the generated proc is exactly the command lambda was called with, a sort of Quines...(RS)This version uses regexp and only works on one variable at a time. Enjoy.
proc apply {body mylist} {
set newlist {}
foreach m $mylist {
regsub -all {\!\!} $body $m cmd
lappend newlist [eval $cmd]
}
return $newlist
}
set mylist {3 43 12 92}
set newlist [apply {expr {2+!!}} $mylist]
puts $newlist
=> {5 45 14 94}
set mylist {fidel vlad paul}
set newlist [apply {format "hello %s" !!} $mylist]
=> {hello fidel} {hello vlad} {hello paul}Okay, the above implementations of lambda all look interesting. I have been using a variant of one of them in a commercial development effort (ah, all those once-off procs resurfaced as lambdas). I am collecting/writing/hacking a package of functional utilities in Tcl. These pages have inspired quite a few of them.So, what would a production ready lambda look like? Apotential summary of some of the lambdas described here could yield:
namespace eval ::lambda {}
proc lambda {arglst body} {
set level [info level 0]
set name [string map {" " _ $ _ \; _ \" _ : _ \{ _ \} _ \[ _ \] _} $level]
proc ::lambda::$name $arglst $body
return ::lambda::$name
}The string map cleans up the lambda proc name so that it is eval safe.-- Todd CoramA generalized map, which takes one or more lists, is at Function mapping.
AM I have been reading a (bizarrely inaccurate) book on Python and I realised that Tcl is filled with lambda-like functionality:
- Commands like [after] and [fileevent] take a script that can be viewed as an implicit lambda.
- All call-backs in Tk are essentially lambdas.
- garbage collection (could be done with ref-counting the generated proc object)
- closure, lexical scoping (there we have a weak point in Tcl)
CL wrote in comp.lang.tcl, on the question whether there are any practical uses for lambda: "Everywhere in Tcl you'd include a script as an argument, Pythoneers might consider a lambda.One of the batteries base Python includes is Tkinter, a binding of Python to Tk. All Tk callback thingies are candidates to be lambdas:
b = Button(text="Push me",
command=lambda: sys.stdout.write("Pushed\n")) "[CL needs to make a point 'bout how the real action in Python is moving dramatically away from lambda and friends, and toward list comprehension.]
CMcC is there any good reason proc returns "" instead of something more useful, e.g. the name of the defined proc, or the body, or even the compiled object? Seems like a waste of a good opportunity to me.
AM I think, no I am positive, that I have found an elegant solution to the problem of an anonymous lambda. It came to me after a weekend and a half of juggling various intricate solutions involving the unknown procedure. In the true spirit of Tcl, I stripped away anything that was redundant, sat down behind a computer and implemented the final version in say half an hour (including testing). Here it is - I do hope I have good enough an understanding of lambda, as it is quite a claim after the above discussion ...
# Anonymous lambda
#
proc lambda {list_args expression} {
return [list LAMBDA $list_args $expression]
}
proc LAMBDA {list_args expression args} {
foreach $list_args $args {break}
return [eval $expression]
}
proc apply {fun args} {
set result {}
foreach a $args {
lappend result [eval $fun [list $a]]
}
return $result
}
proc apply_double {fun1 fun2 args} {
set result {}
foreach a $args {
lappend result [eval $fun1 [list [eval $fun2 [list $a]]]]
}
return $result
}
puts [apply "string toupper" a b c d]
puts [apply [lambda {x} {lindex $x 1}] {a b} {b c} {c d}]
puts [apply_double [lambda {x} {string toupper $x}] \
[lambda {x} {lindex $x 1}] \
{a b} {b c} {c d}] FW: This works fine in some ways, but you're making your own lambda construct that must be called via apply. One of the challenges is to make Lambdas callable like normal procedures, rather than invent a special procedure for managing them. Plus instead of using actual procedures, you're storing them as text and running them via eval, which, for one, doesn't compile the script to bytecode for future use. Also, your method doesn't allow lambdas to retain procedure environment like [info args] etc.Lars H: No FW, lambdas should require an explicit apply in Tcl, because implicit application is unTclish. procs and lambdas should relate to each other as arrays and dictionaries, and there we see that accessing the former happen as part of parsing, whereas accessing the latter requires a helper command. The former have names and exist independently, whereas the latter are simply values (strings) being passed around. The notion that the natural way of calling a lambda is to use the variable in which it is stored where one would otherwise have expected a command names comes from languages where similarly variables are "used" as soon as one gives their name, but such is not The Tcl Way.It is another matter that apply might not be the most striking name for such a Tcl command. My faviourite is (), derived from the traditional infix "apply function" notation f(x) in the usual way of extracting the operation symbols to a separate word. With this one would write proc map {f L} {
set res [list]
foreach item $L {
lappend res [() $f $item]
}
return $res
}FW: Oh, I see, sorry. This is an implementation of passing procedures around as textual lists containing the procedure information. Which is silly, since pass-by-name - with a generated unique name - is good enough as far as I'm concerned, but we're discussing for theoretical purposes here, to show Tcl can do true, passed-by-value Lambdas. I get it now.Still, using eval is a partial solution. There are still scope and state issues. I'd maybe improve somewhat on AM's solution by caching lambdas into real procedures, perhaps using a checksum of the procedure data - a more realistic choice than a "a very telling name", as RS suggested above - then running that procedure instead of using eval, which retains all the benefits of normal procedures. But all of this is theoretical anyway, I'm fine with pass-by-name with generated names.AM I do not mind terribly either if the lambda's are generated "ordinary" procedures with names. It just occurred to me to do it that way, while reading about functional programming. And part of the fun is to explore these theoretical ways.Another thing that occurred to me: The [eval] can be hidden by redefining [unknown] to take care of it :)SS: I proposed Functions As Values for Tcl. The goal is more general than to have lambdas (the idea is to have first class functions), but one of the direct pratical results is that anonymous functions are natural this way.
EB: Yet another version of lambda, already proposed [1]. It is slighly modified here, and the adjuntion of unknown make it simpler to use, like in functions as values. No need of apply or reduce.
# lambda --
# Simply returns a list of a call to lambdaeval
#
proc lambda {paramList body} {
return [list lambdaeval $paramList {} $body]
}
# lambdaeval --
# Does all the work.
proc lambdaeval {__paramList __argList __body args} {
set __argList [concat $__argList $args]
set __paramLen [llength $__paramList]
set __argLen [llength $__argList]
if {$__paramLen > $__argLen} {
return [list lambdaeval $__paramList $__argList $__body]
}
if {$__paramLen < $__argLen} {
# check if last parameter if args
if {![string equal [lindex $__paramList end] "args"]} {
return -code error "too many arguments"
}
set __argList [concat \
[lrange $__argList 0 [expr {$__paramLen-1}]] \
[list [lrange $__argList $__paramLen end]]]
}
foreach __param $__paramList __arg $__argList {
set $__param $__arg
}
return [if 1 $__body]
}
# rename unknown command to handle lamdbaeval call
# without the need of eval
if {![llength [info commands __lambda_unknown]]} {
rename unknown __lambda_unknown
proc unknown {cmd args} {
if {[string equal [lindex $cmd 0] "lambdaeval"]} {
uplevel 1 $cmd $args
} else {
uplevel 1 [list __lambda_unknown $cmd] $args
}
}
}Example:
% set sum [lambda {x y} {return [expr {$x+$y}]}]
lambdaeval {x y} {} {return [expr {$x+$y}]}
% $sum 2 5
7
% set myincr [$sum 1]
lambdaeval {x y} 1 {return [expr {$x+$y}]}
% $myincr 3
4NEM I thought I'd wade in here with my minor variation on the above, using some new features in 8.5a0 (current CVS HEAD). This is quite similar to EB's version above, in that it produces a string rep which is can be eval'ed to provide the result of applying the lambda. This is nice, as it has "free" garbage collection - no proc to explicitly delete. This also means that you don't get the benefits of byte-code compilation, you have to override [unknown] (or use an explicit eval), and there are scope issues aplenty. I can live without byte-compilation, as I'd typically use lambdas in non-speed-critical code (e.g. for quick, throw-away commands). The other issues are more annoying, but could probably be worked around somehow. Anyway, the code: interp alias {} lambda {} list lambdaeval
proc lambdaeval {__arglist __body args} {
lassign $args {*}$__arglist
eval $__body
}
if {![llength [info commands __lambda_unknown]]} {
rename unknown __lambda_unknown
proc unknown {cmd args} {
if {[string equal [lindex $cmd 0] "lambdaeval"]} {
uplevel 1 $cmd $args
} else {
uplevel 1 [list __lambda_unknown $cmd] $args
}
}
}This is very similar to the above, but uses lassign, and doesn't do automatic currying. So: set sum [lambda {a b} {expr {$a + $b}}]
$sum 1 2 ;# Gives 3
set inc [concat $sum 1] ;# Curry, using normal list commands
$inc 4 ;# Gives 5The use of an unchecked lassign doesn't give very good checking for arguments:$sum 1 => error: can't use empty string as operand of "+" $sum 1 2 3 => 3Neither of which are particularly helpful - misleading error message in the first case, and just plain ignoring an error in the second. Also, you can't have default values for arguments, so e.g. "lambda {a {b 1}} {expr {$a + $b}}" wouldn't work.As a further note, in 8.5 you can avoid the overriding of the unknown handler, if you are willing to use the slightly ugly syntax of:
{*}$sum 1 2See also More functional programmingRHS 14August2004 I've been doing some thinking about lambda recently. There's been a lot of discussion about the subject, and I thought I'd share some things I've taken out of it:1. There are a good number of people that think lambda would be useful in Tcl, even if not all those people believe its necessary2. Lambda would be a useful construct (ie, I'm one of the above people). Amoung other reasons, it would clean up code to be able to write
scrollbar -command [lambda {command numOrFrac {type ""}} {
do some things with the commands
}]... rather than having to create a proc for every scrollbar, button, and everything else that takes a command arguement/option. Its become an almost bottled response, when people have problems with getting callbacks to work correctly, to tell them "write a proc, and use that as the callback". Writing a proc for every little thing clutters the codebase, lambda is the perfect solution for that.3. Either:- Lambdas need to be first class objects, and the find the proc that this string represents handler needs to be able to convert a lambda representation to a temporary proc to call (that gets cleaned up afterwards). Or,
- Lambdas need to be handles to stored lambda objects. In this case, garbage collection of Lambdas would be a necessity. If nothing retains a reference to the lambda anymore, there's no point in keeping it around; doing so would be bad in a long running program. The problem is that, if you just clean up any lambda that has its reference count set to 0, you can't do:
set myProc [lambda {x y z} { expr {$x + $y + $x} }
set myProc2 [string range $myProc 0 end]
unset myProc
doSomething -command $myProc2Now, I know that's a very contrived example, but it shows the general idea.4. There are a number of cases where its annoying to have to use ___arg1___, ___arg2___, etc... because you want code to execute in a clean environment. It might be handy to have some kind of procscope command, that worked like uplevel in reverse, creating a clean stack level with args, and running some code in it. For example: set var1 1 ; set var2 2 ; set var3 3
set result [procscope {x y z} { expr { $x + $y +$z } } $var1 $var2 $var3]
# result = 6If we had some kind of procscope, could it be used to create lambda functionality? Probably, but it would still require modifying unknown, and would be slower. On the other hand, if we made lambda accept args on the end, where those args get assigned to the parameters defined, then that might do the job perfectly.In the words of Dennis Miller, "Of course, thats just my opinion. I could be wrong"NEM: See TIPs 187[2], 194[3] and 196[4] which all propose a mechanism to do lambda in Tcl. 194 in particular proposes an apply command which is almost identical to your procscope given above, except that it takes the arglist and body as one argument with an optional namespace specifier.RHS Figures... I thought I remembered at least one TIP that dealt with a lambda construct, but when I looked for them I couldn't find any. That will teach me to take shortcuts and just search for lambda on the TIP page.SS: A pure Tcl implementation of TIP 187 is available in the TIP187 in pure Tcl page.AM Here is a somewhat related issue: Adding, multiplying and composing functions
Dossy Just want to throw this out as it's my preferred behavior:
proc call {script args} { eval $script }Might be used like this:
% set script { puts "[lindex $args 0] [lindex $args 2]" }
% call $script a b c
a cRHS I'm still strongly of the opinion that lambdas should be able to be used in command callbacks, both new and old ones. As such, the following needs to work, imo:
set lamda [lambda {a b} { do something with $a and $b }] ;# or some other way to define lambda
$lambda A BThat, combined with them being anonymous (can't have name clashes with regular procs) and automatically garbage collection, and I'm goldenLars H: Since most callbacks are either scripts or command prefixes, the suggested call will in fact work for them. It's only code that calls callbacks that have to take care to properly {*} the callback.NEM: And seeing as almost always a callback should be invoked via [uplevel], this is not a problem. Regarding call, it should be pointed out that TIP 194 is pretty much identical to that, but with the following improvements:- Formal (named) parameters, rather than positional only,
- Deals with namespace issues (i.e. allows you to specify a namespace),
- Allows byte-code compilation for efficiency.
% set script {{a b c} { puts "$a $c" }}
% apply $script a b c
a cI would very much encourage people to read the three TIPs which have been proposed, along with the archives of the tcl-core mailing list.RHS So, for something like a button in Tk, you would do something like this? button .b -command [list call [lambda {args} { puts "My args were $args }]]If so, that seems reasonable.NEM: Indeed, you can make it even simpler. TIP 194 doesn't propose a constructor command (e.g. the lambda used above), but you can easily write your own:
proc lambda {arglist body} { return [list ::apply [list $arglist $body]] }
button .b -command [lambda {args} { puts "My args were $args" }]and have it even neater.RHS Hmm, it allows for reverse-uplevel too, which I've wanted for quite some time. When I read the TIP, I didn't realize it was so nice. The only downside I see is that it can't be used in the form of: set command [lambda {a b c} { blah }]
$command 1 2 3Are there any others that I'm missing? I'd say thats a significant downside, but every possible implementation will have some kind of downside. I guess the question is, which downsides are we willing to accept, and which not.NEM: Indeed, this is the main point of discussion between TIP 194 and TIP 187 (by antirez). TIP 187 doesn't have this problem, but it requires a change to the evaluation rules (looking for a special "lambda" prefix before normal command lookup). You can achieve the [$command args..] syntax with TIP 194 via a simple unknown command handler which automatically expands the first-word during command lookup. This auto-expansion of leading word would, IMO, be a very nice change to build into Tcl (for Tcl 9), and becomes a very powerful and general mechanism - see TOOT for some of the potential.CMcC The so-far intractable problem has been how to clean up generated procs. Wouldn't [trace add execution $name leave] do the trick? Is it terribly expensive?
proc lambda {args body} {
set name "\[[info level 0]\]"
proc $name $args $body
trace add execution $name leave unlambda
return $name
}
proc unlambda {cmd args} {
rename [lindex $cmd 0] ""
}Sadly though it only works for one invocation.Unless we tinker with [unknown] rename unknown unknown_extend
proc unknown {args} {
set cmd [lindex $args 0]
if {[string match \[\[\]*\] $cmd]} {
set cmd [uplevel 1 [string range $cmd 1 end-1]]
return [uplevel 1 $cmd [lrange $args 1 end]]
} elseif {[llength $cmd] > 1} {
return [uplevel 1 $cmd [lrange $args 1 end]]
} else {
return [uplevel 1 unknown_extend $args]
}
}This works by naming the lambda-generated proc with a bracketed command that regenerates itself ... then detecting forms like [{[script]} arg arg ...] and evaluating the script and treating the result as a command name. I also threw in auto- lead expansion, for the heck of it.At the risk of making it a little too complex, here's a memoizing version which doesn't delete the lambda proc immediately, but only after 10ms, which will let a [map] run without having to regenerate and recompile the lambda. proc ::lambda {args body} {
set name "\[[info level 0]\]"
if {[info procs [string map {[ [[] * [*] \\ \\\\} $name]] == {}} {
proc $name $args $body
trace add execution $name leave ::_unlambda
}
return $name
}
proc ::_unlambda {cmd args} {
set name [lindex $cmd 0]
trace remove execution $name leave ::_unlambda
after 10 rename [list $name] {{}}
}NEM: All three proposed TIPs have no problem with garbage collection (although 196 may be too fragile in the face of GC in my opinion).kruzalex proposed tip for avoiding recompilation without help of using after command. See also the counted package adaptation below too, which returns lappended results after renaming of procName:# first tip
set L {10 20 30 40 50}
proc map {fun list} {
set res {}
foreach i $list {lappend res [$fun $i]}
return $res
}
proc K {a b} {set a}
proc ::lambda {args body} {
set name "\[[info level 0]\]"
if {[info procs [string map {[ [[] * [*] \\ \\\\} $name]] == {}} {
proc $name $args $body
trace add execution $name leave [K [list ::_unlambda] [list rename [list $name] {{}}]]
}
return $name
}
proc _unlambda {cmd args} {
set name [lindex $cmd 0]
trace remove execution $name leave ::_unlambda
}
puts [time {map [lambda x {expr $x*1.16}] $L}]
puts [time {map [lambda x {expr $x*1.16}] $L}]
puts [time {map [lambda x {expr $x*1.17}] $L}]# second tip package require counted
namespace eval lambda {variable unique 0}
proc lambda {arguments body} {
set procName ::lambda::cmd[incr ::lambda::unique]
proc $procName $arguments $body
return [K [::counted::command $procName] [list rename $procName {{}}]]
}
proc unlambda {cmd args} {
set name [lindex $cmd 0]
trace remove execution $name leave ::unlambda
}
proc test {fun list} {
set res {}
foreach i $list {lappend res [$fun $i]}
return $res
}
puts [test [lambda x {expr $x*1.16}] $L]
puts [test [lambda x {expr $x*1.16}] $L]
puts [test [lambda x {expr $x*1.17}] $L]Alexander Galanin <gaa.nnov(at)mail.ru>: My variant of lambdas in Tcl:
proc lambdaLowlevel {paramsVar scriptVar argsVar} {
set params [ uplevel [ list set $paramsVar ] ]
set script [ uplevel [ list set $scriptVar ] ]
set args [ uplevel [ list set $argsVar ] ]
uplevel [ list unset $paramsVar $scriptVar $argsVar ]
for {set i 0} {$i < [ llength $params ]} {incr i} {
uplevel [ list set [ lindex $params $i ] [ lindex $args $i ] ]
}
uplevel [ list eval $script ]
}
proc lambdaProc {params script args} {
if { [ llength $params ] != [ llength $args ] } {
error "Arguments count mismatch: expected [ llength $params ], but [ llength $args ] passed."
}
lambdaLowlevel params script args
}
proc lambda {params script args} {
return [ concat [ list lambdaProc $params $script ] $args ]
}
proc deflambda {id params script args} {
uplevel [ list set $id [ concat [ list lambdaProc $params $script ] $args ] ]
}Usage: set a "hello"
button .b -text "click me" -command [ lambda {text} {
tk_messageBox -message $text
} $a ]or: set a "hello"
deflambda showMessage {text} {
tk_messageBox -message $text
} $a
button .b -text "click me" -command $showMessage