Updated 2012-06-25 06:56:33 by RLE

TCL provided native features to read, to write a pipe, in real time with fileevent or after-the-fact. This solve the need of reading or parsing a command-line output, sending a data to a command and so like.

When needing to handle interactive reading-and-writing, people generally prefer to use Expect for its simplification.

This page is to discuss ways in which pure-Tcl can also used to do some simple interactive read-write where Expect is frequently used. For example, in CVS it is stated that:

...ways to use cvs...
      [open "|cvs -d $cvsroot init" RDWR] and talking to
      cvs along pipes.

Will this really work? If cvs asks for my password, can I really check for that and send it back down the pipe?

Well, yes, you can. If however there is going to be very much interaction, or if the strings being examined are going to be very complex, I (LV) would recommend looking at Expect as a Tcl extension to make the interaction a bit easier.

Pipes exposes some minor troubles, where Expect is always doing as "expected" (except for some cr to lf translations - important when binary stuff is transferred).

Using pipes, fconfigure must be set as follows:
 fconfigure $<pipe> -buffering line -blocking 0

The latter ensures that close $<pipe> does not hang when the pipelined program has an infinite loop (e.g. Tk loop).

Disadvantages compared to using Expect:

  • pipelined program with infinite loop does not close when channel is closed.
  • pipelined program may freeze when stdin is set to send data to a proc via fileevent definition as long as nothing is sent over the pipeline.

Now I can get this to work with something like 'aspell' which is a standalone process, but I find that when trying to do this on 'cvs', that cvs fires up another process (rsh) which wants the password, but Tcl doesn't know anything about that... So cvs is sitting there waiting for rsh which is waiting for a password (I assume -- certainly it is waiting for something!), and since no password is supplied, everything eventually times out and fails.

I understand that for complex uses Expect is obviously the way to go, but if all I want to do is make sure 'cvs' gets my password, it sounds like it would be easy enough just to use pure Tcl. Here's a code outline:
    cd c:/tcl-source/tcl
    catch {
        console show
        update
    }

    proc go {} {
        set pipe [open "|cvs -z5 update ChangeLog" RDWR]
        fconfigure $pipe -buffering line -blocking 0
        fileevent $pipe readable [list piperead $pipe]
        return $pipe
    }

    proc piperead {pipe args} { 
        if {![eof $pipe]} {
            puts "read $pipe : $args" 
            set got [gets $pipe]
            puts "got: $got"
            if {[string first "password" $got] != -1} {
                puts $pipe "my password"
            }
        }
    }

    go

Something like the above can be made to work to interact with 'aspell' to perform spellchecking, but when using cvs, it fires up a separate 'rsh' process which is looking for input, but Tcl doesn't know that, so the above doesn't work.

Any ideas to solve that problem?

I had to solve this today :) Here's what I came up with:
 ---------  cat.tcl  ------------
 fconfigure stdin -buffering none -translation binary -blocking 0
 fconfigure stdout -buffering none -translation binary
 fileevent stdin readable DoIt

 proc DoIt {} {
    global done

    set got [read stdin]
    if {[string length $got]} {
        puts -nonewline stdout $got
        flush stdout
    }
    if {[eof stdin]} {set done 1}
 }

 set done 0
 vwait done
 --------------------------------

 ---------  cvstest.tcl  --------

 set cvsprog c:/dev/wincvs/cvs.exe

 proc tryexec {args} {
    global done

    puts "calling: $args"
    set pipe [open "|$args |& [info name] cat.tcl" r]
    puts "pid(s) [pid $pipe] started... please wait..."
    fconfigure $pipe -buffering line -blocking 0
    fileevent $pipe readable [list gotpipereadable $pipe]
    set done 0
    vwait done
    puts "done!"
 }

 proc gotpipereadable {pipe} {
    global done

    set got [gets $pipe]
    if {[string length $got]} {
        regsub {^(cvs server: )|(cvs\.exe \w+?: )} $got {} got
        puts "*** $got"
    }

    ### check for incompleted lines
    if {[fblocked $pipe]} {
        puts -nonewline "*** [read -nonewline $pipe]"
        flush stdout
    }

    if {[eof $pipe]} {

        set done 1
        fconfigure $pipe -blocking 1
        set status [catch {close $pipe} result]

        if {$status == 0} {

            # The command succeeded, and wrote nothing to stderr.

        } elseif {[string equal $::errorCode NONE]} {

            # The command exited with a normal status, but wrote something
            # to stderr, which is included in $result.
            puts "stderr was: $result"

        } else {

            switch -exact -- [lindex $::errorCode 0] {

                CHILDKILLED {

                    foreach { - pid sigName msg } $::errorCode break
                        # A child process, whose process ID was $pid,
                        # died on a signal named $sigName.  A human-
                        # readable message appears in $msg.

                }

                CHILDSTATUS {

                    foreach { - pid code } $::errorCode break

                        # A child process, whose process ID was $pid,
                        # exited with a non-zero exit status, $code.

                            puts "pid $pid exited with code: $code"
                        puts "stderr: $result"

                }

                CHILDSUSP {

                    foreach { - pid sigName msg } $::errorCode break
                        # A child process, whose process ID was $pid,
                        # has been suspended because of a signal named
                        # $sigName.  A human-readable description of the
                        # signal appears in $msg

                }

                POSIX {

                    foreach { - errName msg } $::errorCode break
                        # One of the kernel calls to launch the command
                        # failed.  The error code is in $errName, and a
                        # human-readable message is in $msg.

                }
            }
        }
    }
 }

 proc DoCvsLogin {} {
    global cvsprog
    tryexec $cvsprog -d:pserver:anonymous@cvs.tcl.sf.net:/cvsroot/tcl login
 }

 proc DoCvsCheckout {} {
    global cvsprog
    tryexec $cvsprog -d:pserver:anonymous@cvs.tcl.sf.net:/cvsroot/tcl checkout thread
 }

 DoCvsLogin
 #DoCvsCheckout
 ---------------------------------

It isn't perfect, but kinda close ;) Note that in the open, |& combines both stdout and stderr together going into cat.tcl just being a pass-through. But I think the trick to the missing prompt was the use of fblocked to check if there is still data in the buffer that hasn't been line terminated yet. There's no \n placed after 'CVSpassword: ', so therefore line buffering is your enemy at this moment.
 D:\bla>c:\progra~1\tcl\bin\tclsh84 cvstest.tcl
 calling: c:/dev/wincvs/cvs.exe -d:pserver:anonymous@cvs.tcl.sf.net:/cvsroot/tcl
 login
 pid(s) 1300 1284 started... please wait...
 *** (Logging in to anonymous@cvs.tcl.sf.net)
 *** CVS password: done!

 D:\bla>

DG