SLB Small Tcl scripts are easy to write with short procedure names placed in the root namespace. As scripts become larger, breaking them into relatively self-contained components which can be re-used and avoiding name conflicts between those components becomes important. This is normally done with namespaces and packages.
While the namespace and package mechanisms work reasonably well, they do add clutter to scripts. [
1] has made it easier to write packages but you do still need to choose a unique name for each package. Sometimes it just seems easier to do something like:
source [file dirname [info script]]/OtherScript.tcl
and bypass the use of packages altogether.
Yet packages have benefits. This page explores an alternative that gives some of the benefits of packages and namespaces but with less clutter and reduced risk of name collisions...
The alternative is based on a 'loader' command which is used thus:
-
- loader use ?-force? ?-cmd cmd? ?-import? moduleName.tcl
Its behaviour is somewhat similar to both 'source' and 'package require'.
The full pathname for moduleName.tcl is obtained (see later). If the module has not already been loaded, the pathname is sourced. This is done within the context of a dynamically allocated namespace. Thus a procedure definition 'proc a {} { ... }' within moduleName.tcl will automatically be distinguished from any other definition of 'proc a'.
Provided the moduleName.tcl is sourced without error (or if it was previously loaded successfully), the commands it exports are made available to the client script. This can be done directly by importing the commands into the client's namespace (if the -import switch was specified) or by adding an ensemble command to the client's namespace that contains the exported commands.
The mechanism for determining the full pathname of a module loaded by 'loader use' is of course based on a search path of directories. However, it does not require a single global search path. Instead, 'loader use' obtains the namespace from which it is called and uses that to determine the full pathname of the file which was sourced when the namespace was allocated. This pathname can have its own unique search path of directories. If the referenced module name is not found within the module's own search path, its parent directory is considered. The mechanism does not look for modules in the parent directory directly, instead it checks whether a search path has been defined for the directory. If it has that search path is used. The search then proceeds through all parent directories up to the top level.
Most directories will have no associated search path, so use of a deep directory hierarchy will not trigger any additional file i/o. Some directories will specify their own pathname as the only entry in their search path. A few may have a search path containing other directories.
Each time a module is loaded, the parent directory is added to its own search path. This is a bit ad-hoc but seems useful.
Modules can be made globally available by placing their directory in the search path for directory '/'. This requires a little special handing on Windows, where pathnames such as 'C:/' are considered top level directories. That special handling is built into the mechanism and so is transparent to its clients.
The -force switch allows a module to be reloaded. In such cases, it is reloaded into the same namespace that was previously allocated for it.
Commands that are not contained within a namespace allocated by 'loader use' can load modules either by specifying the absolute path of the module or by referencing modules accessible from the search path for directory '/'.
The result of all this is that you can create a collection of modules which reference each other using short descriptive names such as Process or Launcher. Such modules can then be placed unchanged into a 'package' (though not a traditional Tcl package) called MyPackage. Clients of MyPackage will reference the modules using names such as MyPackage/Process or MyPackage/Launcher. That larger system could itself be turned into a package, and the original components might then be known to its clients as BigPackage/MyPackage/Process and BigPackage/MyPackage/Launcher.
One limitation of the mechanism compared to traditional packages is that there's no support for version checking. Of course a module can provide a version checking command that clients can call once the module has been loaded.
Some other disadvantages:
- reports in $errorInfo are less meaningful since they reference the dynamically allocated namespaces. We mitigate this by providing 'loader errorInfo', which translates the namespace names into file names. However, Tk's background error reporting uses $errorInfo.
- you can't copy and paste procedures into wish/tclsh console, since procs no longer specify the namespace they belong to.
- module authors must make rigorous use of 'namespace current' or 'namespace code' when defining callbacks.
See also
An Alternative to Namespace for another approach to avoiding hardcoded namespaces.
Implementation edit
# Uses 'namespace ensemble'.
package require Tcl 8.5
namespace eval ::Loader {} {
array set map {}
variable nextId 0
}
proc ::Loader::findFile {clientNs fileName} {
# Find the absolute pathname corresponding to $fileName.
if {[file pathtype $fileName] eq "relative"} {
variable files
if {[info exists files($clientNs)]} {
set clientFile $files($clientNs)
} else {
# Namespace was not allocated by Loader::use, just use the global
# search path.
set clientFile /
}
set child $clientFile
variable searchDirs
while {true} {
foreach dir [getSearchDirs $child] {
if {[file exists $dir/$fileName]} {
return [file normalize $dir/$fileName]
}
}
set parent [file dirname $child]
if {$parent eq $child} {
break;
}
set child $parent
}
if {$child ne "/"} {
# On Windows, need to consider / as a special case since file dirname C:
# yields C:.
foreach dir [getSearchDirs /] {
if {[file exists $dir/$fileName]} {
return [file normalize $dir/$fileName]
}
}
}
error "Cannot find file $fileName in search path for $clientFile"
}
return [file normalize $fileName]
}
proc ::Loader::getSearchDirs {dir} {
variable searchDirs
set dirs {}
catch {set dirs $searchDirs($dir)}
return $dirs
}
proc ::Loader::setSearchDirs {dir dirs} {
# Associate directory $dir with list $dirs.
# When scripts located inside $dir attempt to locate
# a script via 'loader use', the directories in $dirs
# will be searched in the attempt to locate the script.
variable searchDirs
set searchDirs($dir) $dirs
}
proc ::Loader::addSearchDir {markedDir searchDir} {
# Adds a directory to the search path for $dir.
# This is a no-op if $searchDir is already in the search path.
# Hack for Windows, we treat '/' as the root directory, i.e.
# the location for the root directory search path. However,
# in windows, each drive letter is a root directory.
if {$markedDir ne "/"} {
set markedDir [file normalize $markedDir]
}
set searchDir [file normalize $searchDir]
set searchDirs [getSearchDirs $markedDir]
foreach dir $searchDirs {
if {$dir eq $searchDir} {
return
}
}
lappend searchDirs $searchDir
setSearchDirs $markedDir $searchDirs
}
proc ::Loader::errorInfo {} {
# Returned a transformed $errorInfo with dynamically allocated
# namespaces replaced with the associated file names.
variable files
set result ""
foreach line [split $::errorInfo \n] {
if { [regexp { ( *)\(procedure "(::dynamic[0-9]+)::(.*)" line ([0-9]+)\)$} $line dummy space ns proc lineNum]
&& [info exists files($ns)]} {
append result "${space}(procedure \"$proc\" line $lineNum - from file $files($ns))"
} else {
append result $line\n
}
}
return $result
}
proc ::Loader::use {args} {
# Enables use of a module, loading it if necessary.
# -import: imports all commands of the loaded module into the client's namespace
# -cmd cmd: defines cmd as an ensemble through which the client can
# access the commands of the loaded module
# -force force loading of the module even if it was loaded previously.
set extraArgs {}
set mode static
set argNum 0
if {[llength $args] eq 0} {
error "wrong # args: should be \"loader use\" ?switches? fileName\""
}
set fileName [lindex $args end]
set argNum 0
set mode noop
set force false
while {$argNum < [llength $args]-1} {
set arg [lindex $args $argNum]
switch -- $arg {
-cmd {
set mode cmd
incr argNum
set cmd [lindex $args $argNum]
}
-import {
set mode import
}
-force {
set force true
}
default {
lappend extraArgs $arg
}
}
incr argNum
}
set clientNs [uplevel 1 namespace current]
set fileName [findFile $clientNs $fileName]
variable nextId
variable namespaces
variable files
set alreadyLoaded [info exists namespaces($fileName)]
if {$alreadyLoaded} {
set pkgNs $namespaces($fileName)
} else {
set pkgNs ::dynamic$nextId
set namespaces($fileName) $pkgNs
set files($pkgNs) $fileName
incr nextId
}
if {!$alreadyLoaded || $force} {
# We knowthe directory contains at least one module so we make the directory
# part of the search path for all scripts under the directory. This means
# that all modules can load modules in the same directory using the leaf file name.
set dir [file dirname $fileName]
addSearchDir $dir $dir
uplevel #0 namespace eval $pkgNs [list source $extraArgs $fileName]
}
switch -- $mode {
import {
namespace eval $clientNs [list namespace import ::${pkgNs}::*]
}
cmd {
if {![regexp :: $cmd]} {
# Normal case, client has specified the command within its namespace,
# so concatenate them together.
set cmd ${clientNs}::$cmd
}
# else client has specified the cmd name within the namespace hierarchy.
namespace eval $pkgNs [list namespace ensemble create -command $cmd]
}
noop {
}
default {
error "Internal error - unknown mode $mode"
}
}
}
namespace eval ::Loader {
namespace export {[a-z]*}
namespace ensemble create -command ::loader
}
'''Example usage'''
source Loader.tcl
loader addSearchDir / [pwd]
loader use -cmd m1 SubDir/Module1.tcl
m1 a
loader use -cmd m2 Module2.tcl
proc a {} {
puts "This is proc a from Module1 defined in namespace [namespace current]"
m2 a
}
proc b {} {
puts "This is proc b from Module1 defined in namespace [namespace current]"
}
namespace export {[a-z]*}
# Note: Module1.tcl and Module2.tcl both use each other. This is not
# necessarily desirable but illustrates the flexibility of the mechanism.
loader use -cmd m1 Module1.tcl
proc a {} {
puts "This is proc a from Module2 defined in namespace [namespace current]"
m1 b
}
namespace export {[a-z]*}
