NEM OK - here's an attempt at explaining {expand} in Tcl 8.5. The easiest way to explain is perhaps with an example. Consider the following code, which may be used for launching a browser on Windows:
eval exec [auto_execok start] [list $url]The reason that the [eval] is there is that [auto_execok] may return a list which represents the command to be run. This list needs to be "flattened" for exec. For instance, let's say [auto_execok start] returned:
% auto_execok start c:/windows/command.com /c startIf you just did:
exec [auto_execok start] $urlYou would get an error about "can't find 'c:/windows/command.com /c start' - no such file or directory" or something similar. In order to flatten this list, Tcl before 8.5 requires use of [eval] (or more arcane uplevel equivalents) which does this for you (as it concats its arguments together before evaluation).So, why not just say
eval exec [auto_execok start] $urlBecause the $url may contain spaces, so we need to protect it so that it doesn't get flattened. So, for pre Tcl 8.5 programmers, the safe way to code is via the use of [list]:
eval exec [auto_execok start] [list $url]There are lots of gotchas with quoting arguments correctly for [eval]. So, to solve this problem, 8.5 introduced the {expand} syntax, which inline expands arguments in a command invocation. What this means is that {expand}list becomes word word word... at invocation time. So, we can now rewrite our example (in 8.5) as:
exec {expand}[auto_execok start] $urlwhich is clearer, safer (by avoiding eval) and better all round.RS: {expand} is, so to speak, the inverse of list. It is not a command (because every command returns one single word), but an extension to the Tcl syntax. Consider:
{expand}[list 1 2 3]produces the three words1 2 3And for using expand, it is good to look out for places where eval is used:
eval pack [winfo children .]is equivalent to
pack {expand}[winfo children .]But for code to be given to others (which might still run 8.4, 8.3 or older), best avoid {expand} for a while...LES: Would someone remind us again why this command had to be braced? Why {expand} instead of expand? Why not [ expand [ list 1 2 3 ] ] ?MG - I think it's because expand isn't actually a command, like [list], or [pack], etc; it's more of an 'option'. If you used [expand [list 1 2 3]] you'd get an 'invalid command name' error. It's a little confusing, I think, since it seems to be different to all other aspects of the Tcl syntax. . .DKF: Actually, it's because expansion is not command-like behaviour (i.e. the result of a command is never broken up into multiple words by the Tcl interpreter itself.) Allowing commands to specify that they return an expanding list (and no, I don't know of any way to limit the effects to just some magical expand command) causes all sorts of problems, potentially including security faults when handing values between interpreters. And it actually makes the Endecalogue much more complex than the {expand} version, as it requires a lot more complexity to be added to the existing rules. By using a new piece of core language syntax, we side-step these problems and focus on giving simple expansion control to the programmer at the point when the programmer actually wants it (you could do it before with eval but it was very error-prone.)SLB: The syntax was controversial. One discussion of the subject is recorded in DISCUSSION: Argument Expansion Syntax. See especially the comments from MS towards the end.Because backward compatibility (a virtue to be sure) overrode aesthetics (a higher virtue, IMO).Hmmmmm. Those who find
... [expand [list 1 2 3]] ...superior aesthetically should, CL believes, re-read the discussion and Miguel's comments. {expand}'s introduction is not some sort of reactionary defense of abstract compatibility principles; instead, it preserves valuable syntactic simplicity.I agree that the functionality warranted new syntax. It's just that the chosen syntax is exceptionally ugly, and it seemed that pretty much everyone agreed that it was ugly, but it was backwards compatible. But since I've engaged in this argument before and lost, there's no point in having it again. It's also the reason why I'm not putting my name on this.Although, given that Tcl's big claim to fame is having almost no syntax (Tcl: Just 11 syntax rules!), I'm still curious as to how a 12th syntatctic rule (Tcl: Now with almost 10% more syntax!) makes it into a minor revision."... with difficulty", CL judges. Was it the right thing to do? *I* won't be secure in that until we have a few years' of experience with it.MG My two cents :) I think it's an awful shame that something so against-the-grain had to be added to make this possible, but I do definitely see the uses for it (even though I've not used it myself, still being on 8.4). As far as I can see, there was absolutely no way to achieve this effect without a change in syntax; I personally think that it's worth it, for all the hassle it saves with quoting for eval, etc.DKF: FWIW, I believe that a sub-optimal syntax was selected and that it should have been done as part of Tcl 9.0 (but I didn't really want to see 8.5.) But other people disagreed with me on these matters, and I didn't (and don't) want to reopen those arguments (if you do, shout at Jeff :^). I will absolutely stand by the semantics of {expand} though; that's exactly right.CMcC: IMHO {expand}$x is simply ugly (or, rather, complexly ugly), and sticks out like a wart on the otherwise flawless body of tcl :) I would have preferred a composite brace style, such as {$a}@ or similar or grave accent: $a.LES: Ouch. What a can of worms I have opened. I was afraid it could happen. ... Honestly, I am pretty confident that the remarkable amount of heated debate over this intriguing issue by so many smart and experienced senior geeks has resulted in the most appropriate solution. However, I have read a lot (including lots of c.l.t., but not all) of that discussion and still can't find answers to this little idea that tickles the back of my mind: {expand} is mostly intended to solve a problem that almost invariably affects exec, isn't that right? Didn't someone suggest exec -expand when this discussion started? What was wrong with that idea? I don't remember ever seeing the argument that dismissed it.RHS: In the way of a minor answer to the above, the ability to expand a list into separate arguments comes up in a variety of places, not just with exec. To add to that, a -expand flag would not totally satisfy the situation, even if it existed for every command. Keep in mind that {expand} allows one to specify which lists need to be expanded, rather than having all arguments expanded.LES: Thanks. And what was wrong with creating an [expand $list] command? Or maybe concat -expand or concat -recurse?
% concat hey {{1 2 3}} {a} {{{aa ab ac} {ba bb bc}}} mom
hey {1 2 3} a {{aa ab ac} {ba bb bc}} mom
% concat -recurse hey {{1 2 3}} {a} {{{aa ab ac} {ba bb bc}}} mom
hey 1 2 3 a aa ab ac ba bb bc mom[aotto]: does this meat that !every! argument have to be checked on {expand} ?? I think this is very slow because you check ~thousend of words to get one {expand} I vote for a expand comand and a new obj type expandList which acts like {expand}DKF: The current behaviour is actually very fast and causes no performance degradation at all when not used (the parser is slightly more complex, but only in what was previously an error-case). It's also been perf-analyzed so as to get an implementation technique that balances speed with maintainability. (The cost of looking for {expand} is tiny because Tcl already has to look at all those characters.)FWIW, I wanted $x/[x y z] but that was a 9.0-only solution and many people didn't want to go to 9.0 for various reasons.WHD: Is there any reason that your "`$x" notation couldn't be adopted as a synonym for "{expand}$x" in 9.0? It's certainly nicer to look at.RS: Gentlemen: all hopes for an [expand] command are vain, because every command returns one word as result, while {expand} will result in zero or more words. This requires a syntax change in Tcl - it just can't be done with the model we have so far. Nobody will be forced to use {expand} - it's up to each coder whether he prefers eval or {expand}.LES: Many commands return a list. Good enough, IMEHO. So long as it's ONE single list, with no other lists in it.RS: Sure enough, but a list is one word. The problem arises when you want a list to be spliced, e.g.
file copy [glob *.tcl] targetdirThis case requires either eval or {expand} (if you don't have one single Tcl file :)
Why should an individual developer {expand} rather than [eval], when the former loses so much version compatibility? Donal leads off with an apt example:
set retList {
foo bar
[exit] $spong
}You can make this work with eval,
eval kooProc [lrange $retList 0 end]but it's unsightly and inefficient of cycles.Lars H: A better eval form is to use linsert:eval [linsert $retList 0 kooProc]This avoids the shimmering that is the reason for the above inefficiency. (Some people may of course criticise it for being even more unsightly.)
Sarnold Some funny stuff we could do with expand (IMHO it could work properly):
set btprop [list -background #aaa -foreground #ffeeaa -relief flat]
button .button1 -text "Proceed" {expand}$btprop
pack .button1
button .button2 -text "Blah" {expand}$btprop
pack .button2The code above could be a quick and dirty way of creating styles in your GUIs. (just for fun)SS 11Jan2005 - I see there is some idea about adding as alternative syntax of {expand}. I think it's very important to select just one way to do it. What is better between this two? I think, {expand} is better. We are not generally going to use {expand} every ten lines of code, it's a useful but relatively rare syntax, so to be clear and explicit wins over to be short. But then, we may wonder if there are alternatives that are better than both {expand} and . Maybe there are: for example {} is not as subtle as , it's visible enough, still not as verbose as {expand}. {} is less explicit, but after all, no one will get the sense of {expand} just looking at it... (my concern against is that it is hard to see if you look at the source code), on the other hand {} scales well against a future where we will find that, against our previsions, we are using {expand} a lot in normal code. So I vote for {}, but my vote counts as zero :) If you agree with me, say it here or try to express your opinion in the chat with TCT members. Also I'm not sure if we are still in time to change it or not.[JDW] 11Jan2005 - I think {expand} syntax is fairly ugly, but the semantic content is very useful. How about some variation on ${listVar} like $@{listVar} or perhaps ${@listVar}? If "@" is a problem, pick another symbol. If ${...} is a problem, how about $(listVar) or ^{listVar} or similar?SS the problem is about listVar: {expand} does not work just against variables, but against everything is a single argument. For example this is valid Tcl8.5 code:
{expand}{puts hello}DKF: Semantically and structurally, {expand} is right on the money. It does exactly what is needed and is implemented exactly right. Syntactically, it sucks.{expand} seems conceptually related to lisp macros, something like ,@ Could this syntax be made extensible and more general-purpose? Not that it would really be useful for much, but I could see {eval}$foo being a synonym for [eval $foo], for example. Likewise, I could see uses for lisp style read-macros or forth style immediate words, but certainly not before 9.0.MG Aug 15 2004 - Assuming you mean something like set foo [list puts -nonewline [list "My message"]]
{expand}$fooI think it would work anyway, doingputs -nonewline "My message"Is that what you mean?escargo 16 Aug 2005 - I was thinking that {expand} reminded me of a Forth IMMEDIATE word, one that gets applied before other transformations. Commands only return a single thing, while {expand} removes one layer of list wrapping. As the example using for below shows, the results are not what one always expects because each command is free to interpret its arguments as it chooses.
MG I'm still using Tcl 8.4, so can't check, but what would 8.5 do if you have, for example,
set someVar {expand}Would it do The Right Thing, and actually set the variable to expand, or would it expand nothing and set the variable empty? And would set someVar {{expand}}be a reasonable work-around to it, or would that effect the nesting-depth (in terms of lists) when the var was set? Or would it still try and expand? In reply to what SS said before, maybe trying to use some characters that aren't used elsewhere would be better? Off the top of my head, I can't think of anywhere Tcl gives any significance to ~ or - maybe expand (rather than just ) might be more user-friendly, without confusion of where "{expand}" is a braced string and where it's the new syntax?DKF: Both those examples would work exactly as in 8.4 and before. The expand syntax is like this: set someVar {expand}$varContainingAListTry entering that in 8.4 and see why the syntax was chosen... ;^)MG *grins* In that case, it sounds good to me. It's a real shame to have to change the syntax, IMHO, but well worth it for what we'll get out of {expand}. Now I've just gotta wait for 8.5 to be released and used commonly so I can start using it... :)LES on August 15 2005:
% set var {{a b c} {d e {foo bar hey} f} {g h i}}
{a b c} {d e {foo bar hey} f} {g h i}
% foreach i $var {puts $i}
a b c
d e {foo bar hey} f
g h i
% foreach i {expand}$var {puts $i}
a
b
cUh?! I REALLY don't know what just happened here. :-(MG I think (I'm on 8.4, so this is a best guess) that the problem is that the {expand} here basically turns this into:
foreach i {a b c} {d e {foo bar hey} f} {g h i} {puts $i}Which takes advantage of foreach's other syntax,foreach $varlist1 $valuelist1 $varlist2 $valuelist2 $commandsSo, it sets $i to each of a, b and c, each of $d, $e, ${foo bar hey} and $f to g, h and i.The correct way to do what you want would probably be (but untested, as I don't have 8.5)
foreach i [list {expand}$var] {puts $i}A/AK: Using sugar for forward-compatible {*} may be of interest for those who want to give {expand} a try without upgrading to Tcl8.5.
IL Whether or not it would have been possible, it simply destroys the tcl aesthetic, and frankly opens the door for future "ugly" inclusions into the language imho. I strongly suggest some rethinking towards this.
interp alias {} eval {} expandsIL, do you realize that the above isn't even close to a proper replacement for {expand}? I also whether this opens the door for 'future "ugly" inclusions'. As much debate as this issue generated, I doubt anyone is anxious to go through that again.Oops, I had it backwards. I just did a little reading on how old this controversy is, so I'll stop there. Sorry it ended up this way, for what it's worth.07nov05 jcw - Warning: can-of-worms ahead!I'd like to see "{expand}..." dropped from 8.5, and replaced by the following three conventions instead:
- {*}... <= expands
- {#}... <= shorthand for [llength ...]
- {x}... <= shorthand for [lindex ... x], with x only <int> or "end-<int>"
See also:
