Features of a software library system

A software library is a software engineering feature, rather than a language feature. However, most languages have some support for grouping sets of related functions into a single entity.

Early Tcl had no such facility, beyond ad-hoc techniques of placing related source code modules in a subdirectory and using the Tcl's auto loading facility.

[LV well, early tcl had source for files of Tcl procs and load for C code. And there was , as is mentioned later, the tclIndex code].

The package command was added with Tcl version 7.5 in 1996. This command piggybacks onto some of the features already present in the auto load functionality (using auto_path for instance) to allow for grouping files into a package and automatic loading of packages. Source code modules can be defined a members of a library with the package provide and pkg_mkIndex commands, and may be loaded with the package require command.

Release 8.2 and 8.3 modified the default behavior of the package command to make it more independant of the auto_load implementation. Instead of simply finding packages and populating the auto_load infrastructure, the package require now loads the modules immediately.

This table describes some of the features of Tcl/Tk software libraries. These include requirements that a system should support, and characteristics of different types of libraries that should be supported. [1]

Some requirements are slightly different depending on whether the item in the library is a Large item - something you would build an application from (the http package) or a Small item - something you would build a larger subsystem from (a tree data structure, or complex widget). The requirements for a compiled package, like the incr Tcl extension are different than for a pure Tcl package. Other requirements are shared by both Large and Small libraries.

There are three phases of library use.

 1. Pre use: Library construction

 Members of a library must be grouped in a logical collection.
 An identifying name must be assigned to the library.
 Revision information must be assigned to the library.

 2. Runtime: Library identification

 The application must be able to find the required library.

 3. Runtime: Library loading

 The library must be loaded into the application either with load or source.

 4. Runtime: Introspection

 Determine if a library is loaded, and if so, what revision is loaded.

For example, code may prefer to have version x.y of a library, with workarounds if version v.w is found instead.

Many of the requirements described above are met with the current package command, however the package command predates several modern Tcl language features (namespaces and vfs), and was designed to support large scale libraries - compiled extensions or large Tcl-source collections (ie. http, or DRH's Tree widget) that are designed to be built into an application.

There is not good support for small scale libraries of modules designed to be built into larger subsystems, which are then built into applications. A stack or tree data structure, or a complex widget with internal state are examples of this scale of library.

The package command works well for Large style entities that reside on a locally mounted disk.

The package command does not support finding libraries on unmounted file systems, or small scale libraries.

The current package command does not support these file finding needs:

 1. Finding libraries on any platform except the locally mounted file system.
 2. Finding a library with a complex matching rule (ie. A revision that is more than X but less than Y)

The current package command does not support these Small mode program features:

 1. Load library into an application specified namespace.
 2. Loading multiple copies of a library.

Options

 1. Extend package command to support the unmet requirements.
 2. Create a new command similar to package to support Small mode development needs.
 3. Create a new set of library commands to support all the requirements of package.

    In this case options include:

   3.a Separate the library creation, library loading and library introspection functionality into separate commands.
   3.b Create a single command (and subcommands) to implement all the functionality.

   3.c Develop external support for software libraries, as the C compiler does with the ar and ld commands.

    3.c.1 Use a wrapping mechanism similar to prowrap, sdx or freewrap to create single file execution modules.
    3.c.2 Use external Tcl script to generate a Tcl Library, similar to a .a, .dll, .so or .jar file, and add a command to the Tcl core to duplicate package require or #include functionality.
    3.c.3 Use an external Tcl script to create separate index files (As pkg_mkIndex does now), and a more generic package require command implementation.

Comments on or edits to this page are welcome.

---

schlenk One should mention the new Tcl modules, which were introduced in Tcl 8.5 in this context, which address some of the issues, especially from 3.c.

Loading multiple copies of a library (in a single interpreter) is a rare requirement from my POV. What's the use case for it; it just sounds like a waste of memory. Loading a library into an application specified namespace is not trivial. It is more common to load the package and import the exported API via namespace import into the namespace one wants.

I don't really see why the current package mechanism is seen as not useful for small mode development. What's wrong for example with the tcllib struct packages which fit into your proposed small mode?

---