Summary edit
A historic document for reference from other sites.Philip Quaife 2005-03-27:Description edit
This page contains a definition of the change requirements to the Tcl core to eliminate the problems that developers have in distributing and deploying their code.Introduction edit
In the present time, end users have no clue about installing software. The analogy I use is that of people driving cars. People expect a car to just work without having to understand what goes on under the hood.The past and present view of core developers is that end users should understand how to operate Tcl in order to use the applications that have been developed. Note that the word application is used here in its wider meaning.The end user has a Click and Run mentality, if the process is more complicated than that they don't want to know.The Present Situation edit
The end user locates and downloads a Tcl application. They then find it does not run. Either Tcl is not installed or it is the wrong version.The end users are then required to locate the Tcl interpreter, download, compile, and install it. They then locate, download, compile, and install any extensions that are needed by the application they wish to run.Most users have given up well before the end of that process because they either lack the skill or inclination to do the required work.Attempted Solutions edit
Skipping a lot of pain and suffering by developers (at the hands of the cruel TCT overlords), we now have two solutions, starkits and Freewrap.The deficiency in both of these is illustrated in the following example.- DeveloperA creates tclkit and makes it available for common platforms.
- DeveloperB uses tckit and makes an application, puts it on the internet and anounces 'look at me, look at me'.
- UserC downloads the kit and it does nothing. So they email developerB for help.
- DeveloperB directs the user to DeveloperA for the appropriate tclkit to run the application.
- DeveloperA informs UserC that there currently is no tclkit for their platform (platformX) and to see DeveloperB to make one.
- UserC goes looking for another application and now has the view that Tcl and DeveloperB suck.
Whose task is it to create a tclkit/freewrap for platformX ? edit
DeveloperA does not have access to the hardware platform, UserC has the hardware but can't program, DeveloperD has the hardware but does not understand the build process for tclkits and can't resolve any build failures.No matter which way you try and solve the problem we always end up in a Catch 22 or Mexican Standoff position.The Solution edit
The failure in the process is that in order to release binary files for the users' convenience we must create a custom Tcl interpreter. A task which is beyond most developers.The question then is , what changes to the core are required to allow the above to happen without creating a custom tcl interpreter that will operate as a stand alone executable.These are the minimum changes required.- Make Tcl boot process detect that data has been attached to the end of the tcl interpreter executable file.
- Make the binary data accessible at the script level.
- Source the binary data.
STARPACK MYTHICAL
|-----------| |------------|
| custom | | standard |
| tcl | | tcl |
| interp | | interp |
| + | |------------|
| vfs code | |marker|
| + | |------------|
|extensions | |boot script |
| + | |^Z|
|init code | |------------|
|-----------| | binary |
| binary | | data |
| data | |------------|
|-----------| |marker|
|marker|A Simple Example edit
To demonstrate the boot process lets take a simple example to say play a sound file using snack:# Boot code set f [open tmpfile w O_BINARY] puts -nonewline $ [embed payload 0 187645] close $f load tmpfile snack snack::play $argv exit ^ZTo understand the above, we assume that the standard Tcl interpreter includes an [embed] command that performs as follows:
: '''embed payload''' ?''start''? ?''end''? : allows access to the attached data, starting with the first character after the ^Z.''
Other subcommands to [embed] will be outlined later in this document. The above however is the minimal new command necessary.
Practical Issues edit
The binary data is likely to be quite large and so we would like to compress it to reduce transfer time and such.We can do this without any change to the core if we make our file as:| tcl interpreter | | marker| |boot script ^Z | |inflate dll | |rest of data | | marker|The boot script would become:
# Boot code set f [open tmpfile w O_BINARY] puts -nonewline $ [embed payload 0 15345] close $f load tmpfile inflate set f [open tmpfile w O_BINARY] puts -nonewline $ [embed inflate [embed payload 15346 234567]] close $f load tmpfile snack snack::play $argv exit ^ZAny modern decompression algorithm is likely to be less than 10K of binary code so it is not a necessity to include such code inside the core itself. By not including one we can also take advantage of newer more efficient compression algorithms as they are developed without having to recompile Tcl or requiring the end users obtain a newer version of Tcl.
Cross-Platform Compatibilty and Trade-offs edit
While any decompression routine would be small enough to attach to a payload we may still have the problem of getting a compiled decompression library for every platform that the Tcl interpreter can be compiled for.It then makes considerable practical sense to include an inflate type command inside the core as an exension library so that it will be built as part of the normal Tcl build process and thus will automatically be available wherever Tcl is. By building it as an extension the developer is free or include something better but always has the Tcl extension as a fall back position.Another issue is the loading of shared dlls. Some platforms do not support this, while others can load an extension directly from memory.This makes it a practical necessity to provide a boot process extension loading mechanism outside of the normal Tcl load command. On most platforms it is likely that the normal code will be used, but for those platforms that don't we can implement a platform specific solution. We could call this command:
embed load ?start? ?end? #''This loads an extension straight out of the binary data''.Another requirement is to free the developer from having to maintain binary versions of their applications for end users to download. There are currently projects under development that are designed to handle this deployment issue so the solutions will not be outlined here.Planned Core Changes edit
It is planned that the core will include a number of extensions such as [zlib%|libZ], which are aimed at addressing the issues raised here. Unfortunately the problem is more basic than the issue of core features and libraries. The issue is in initialization and startup of the interpreter which defeats the present day developer.Conclusion edit
The minimal changes required to the core are:- Detect an attached payload, defer initialisation and source the binary payload.
- Add a command to retrieve the binary data from script level.
Practical extensions to the core could be:
- A decompress command.
- Load an extension directly from the binary command.
Misc edit
NEM: What common platforms are missing a tclkit?PWQ: What is defined as common?Since Microsoft Windows has 95% of PC market I guess that tclkit has it covered.Assuming that your question has more meaning than the literal, from equi4 web site:- 8.5a4 6 platforms
- 8.5a2 11 platforms
- 8.4.12 2 platforms
- 8.4.4 9 platforms
- 8.4.0 19 platforms
- mobiles 1? , pda 2?
Twylite: The introduction states that "the end user has a Click and Run mentality". This is important for context, as end users also understand "install and run". As such this page addresses only single-EXE binary distributions (download, click and run).With an install-then-run approach you can separate the distribution into separate files that will be included in an install package (i.e. setup.exe or msi on Windows). In such a case custom start-up logic still helps in order to keep the install structure neat (e.g. put Tcl's libraries and init script into a VFS) and avoid the need for batch files (clicking myapp.exe could run myapp.tcl, rather than creating myapp.bat to run "myapp.exe myapp.tcl").
