knit

knit, by PYK, is another macro facility for Tcl.

Synopsis edit

: auto cmd ...
: knar body
: knit name arguments ?process? body
: knead arguments ?process? body
: util auto varnames script

Download edit

knit is also available as ycl::knit::knit, along with unit tests.

Description edit

knit is useful for those times that you want something like eval, but with the ability to programatically manipulate the script to be evaluated. It creates a procedure that when run, makes substitutions to body and the evaluates body at the caller's level. process, if provided, is a script evaluated at the local level before body is processed and evaluated body. process provides a sandbox in the form of a local procedure in whose scope macro variable and command substitutions are processed.

knit uses tailcall to provide some of the features that were problematic in earlier macro systems. Each macro is a procedure that fills out a template according to the arguments it receives, and then tailcalls the template.

knit takes an EIAS approach to macros, meaning that it does not try to discern the structure of the template it is filling in, and instead provides the macro author a convenient syntax to choose how subtitutions are made. It turns out that just a small number answer most needs. All macro substitutions happen textually. They do not respect the syntactical flow of the Tcl script. It's the responsibility of the script author to make sure the macros produce a syntactically correct script.

knead can be used to build a macro procedure specification without actually creating the macro procedure. knit is implemented as a trivial wrapper around knead. knead itself is useful for creating anonymous macros:

apply [knead x {expr {${x} * ${x}}}] 5

In turn, knot is a simple wrapper around knead that also executes apply.

knar performs only macro command substitution.

auto generates the arguments to the specified command from variables of the same name in at the caller's level. If varnames is the empty string, auto also generates varnames from body.

In contrast with Sugar, knit is more interleaved with the running interpreter, as Lisp macros are. Where Sugar attemps to parse a script and discern macros, knit inserts the macro code at runtime when the macro procedure is invoked. In order to do its expansions, Sugar must know, for example, that the first argument to while is evaluated as an expression. knit is oblivious to such things, allowing it to fit more naturally into a Tcl script. Since knit macros are themselves procedures, knit eschews the issue that {*} raises for Sugar, and in general automatically has the features of a procedure that the merely-procedure-like macros in Sugar have to work hard for. One example is default arguments and another is $argv handling. The tradeoff is that knit incurs some cost during runtime that Sugar does not, namely the cost of the tailcall.

util auto simply performs variable macro substitutions in a script and returns the script. If varnames is not the empty string, ''varnamess' is derived from script. substituted values are retrieved from the level of the caller.

Macro Substitutions edit

${arg}: Replaced by the value of $arg, properly escaped as a list.
#{arg}: Replaced by the value of $arg, without escaping the value as a list. This is useful for example, to substitute a fragment of an expression into expr, or to substitute a few lines of code into a routine. Also useful for substituting in a command prefix.
!{argname}: This is simply for convenience, and is exactly equivalent to [set ${argname}]. In other words, the value ${argname} is the name of a variable, and it will be arranged for the value of that variable to be substituted at execution time. In the examples below, lpop2 does the same thing as lpop, but thanks to !{argname} is a little more concise.
[`name ...]: Replaced by the returned value of command macro named name. By default, the replacement value is rescanned for additional macros until all macros are expanded. Useful to perform more complex and arbitrary substitutions. This happens prior to the macro variable substitutions so that any variable macros in the substituted text are still processed as usual later. The standard Tcl substitutions are performed, at the level of the process script. This gives command macros access to any variables defined by that script.

Command Macros edit

addvars varnames: Each varname in varnames is added to the list of macro substitution variables to process.
def name args script: Create a macro named name that substitutes each arg from args, into script, as described for knead.
defdo name args script ?value...?: Performs def, and then do, passing it the value arguments.
do name ?value...?: Executes the macro command named name, passing it all the value arguments.
eval script: script is evaluated at the level of the process script, and the returned value is the value of the macro command.
foreach varname list ?varname list ...? script: Each list. Like foreach, but each set of values extracted from the lists is is assigned to the corresponding varname names, and macro variable substitutions in script are processed against these variable names. This is useful for inlining commands or producing nearly redundant code from boilerplate, taking advantage of byte-compilation of procedures.
if ...: Like if, but triggered body arguments are simply returned.
script script: Like eval, but the value of the macro command is the empty string.

Configuration edit

The following variables can be set to configure the behaviour of knit and friends:

knit::knarname: The string that, when preceded by [, indicates a command macro. The default is `.
knit::recursive: A boolean value that indicates whether command macros should be recursively processed.

Customizing edit

To create a customized knit, use ycl::dupensemble to duplicate the ycl::knit ensemble, and then add commands to cmds child namespace of the namespace of the new ensemble. A conforming macro comand accepts on argument, cmdargs, and returns a value the is to be substituted.

Examples edit

These examples show how the macros presented in Sugar, along with various other macros are implemented in knit

knit unit tests: Toy examples.
ycl::chan: Uses the foreach macro to substitute some boilerplate code.
lswitch: The most extensive example yet. Uses knit to implement a switch for lists.

More examples:

knit double x {expr {${x} * 2}}

knit exp2 x {* ${x} * ${x}}

knit charcount {x {char { }}} {
    regexp -all ***=${char} ${x}
}

knit clear arg1 {unset ${arg1}}

knit first list {lindex ${list} 0}

knit rest list {lrange ${list} 1 end}

knit last list {lindex ${list} end}

knit drop list {lrange ${list} 0 end-1}

knit K {x y} {
    first [list ${x} ${y}]
}

knit yank varname {
    K [set ${varname}] [set ${varname} {}] 
}

knit lremove {varname idx} {
    set ${varname} [lreplace [yank ${varname}] ${idx} ${idx}]
}

knit lpop listname {
    K [lindex [set ${listname}] end] [lremove ${listname} end] 
}

knit lpop2 listname {
    K [lindex !{listname} end] [lremove ${listname} end] 
}

foreach cmdname {* + - /} {
    knit $cmdname args "
        expr \[join \${args} [list $cmdname]]
    "
}

knit sete {varname exp} {
    set ${varname} [expr {#{exp}}]
}

knit greeting? x {expr {${x} in {hello hi}}}

knit until {expr body} {
    while {!(#{expr})} ${body}
}

knit ?: {cond val1 val2} {
    if {#{cond}} {lindex ${val1}} else {lindex ${val2}}
}

knit finally {init finally do} {
    #{init}
    try ${do} finally ${finally}
}

Sometimes only the macro command preprocessing is wanted. Using [knar] alone is rather like runing a C file through the preprocessor. Here's an example:

proc p1 {some arguments} [knar {
    [` foreach x {1 2 3} y {4 5 6}  {
        set coord#{x} ${y} 
        lappend res $coord#{x}
    }]
    # There is no variable named "x" in this scope at runtime.  Macro
    # expansions operate in their own sandbox (a local procedure in which the
    # script to be evaluated is generated. 
    return $res
}]

p1 ;# -> 1 4 2 5 3 6

Example: Avoid a Conditional Branch in a Loop edit

Sometimes only one or two steps of a routine branch based on some condition. It can be annoying when one of those steps is in a loop, and the condition must be tested on each iteration even though the values affecting the outcome are known prior to entering the loop:

proc files {arg1 arg2 arg3} {
    ## step 1

    #step 2
    if {$arg1} {
        #do something
    } else {
        #do something else
    }

    foreach file $files {
        #step 3

        #step 4
        if {$arg1} {
            #do something else
        } else {
            #do something else
        }
        #step 5
    }
    #step 6
    #return some result
}

In this situation, knit could be used like this:

proc files {arg1 arg2 arg3} {
    ## step 1

    if {some condition} {
        #step 2
    } else {
        #alternate step 2
    }

    if {some condition}
        files_for {
            #the script for step 4 
        }
    else {
        files_for {
            #the alternate for step 4 
        }
    }

    #step 6
    return files
}

knit files_for script {
    foreach file $files {
        #step 3

        #step 4 is a macro
        #{script}

        #step 5
    }
}

Or, if the conditions can be determined from just the parameters to the procedure, multiple variants of the procedure can be generated from a template, and the selector moved to the caller:

knit files_macro {name script1 script2} {
    proc ${name} {arg1 arg2 arg3} {
        ## step 1

        #step 2 is a macro
        #{script1}

        foreach file $files {
            #step 3

            #step 4 is a macro
            #{script2}

            #step 5
        }
        #step 6
        return files
    }
}

files_macro files1 {
    #script1
} {
    #script2
}

files_macro files2 {
    #script1
} {
    #script2
}

if {$arg1} {
    files1 $arg1 $arg2 $arg3
} else {
    files2 $arg1 $arg2 $arg3
}