CODEWITHSUNDEEP
tclfork
Gert Gottschalk
Gert Gottschalk

Reputation: 1716

Time resolved memory footprint of TCL exec

What's the high resolution time axis behavior of TCL 'exec ' ? I understand that a 'fork' command will be used which will at first create a copy of the memory image of the process and then proceed.

Here's the motivation for my question:

A user gave me following observation. A 64 GB machine has a TCL based tool interface running with 60GB memory used. (let's assume swap is small). At the TCL prompt he gives 'exec ls' and the process crashes with a memory error.

You insight is much appreciated. Thanks, Gert

Upvotes: 1

Views: 744

Answers (1)

Donal Fellows
Donal Fellows

Reputation: 137627

The exec command will call the fork() system call internally. This is usually OK, but might run out of memory when the OS is configured to not swap and the originating Tcl process is very large (or if there is very little slop room; it depends on the actual situation of course).

The ideas I have for reducing memory usage are to either using vfork() (by patching tclUnixPipe.c; you can define USE_VFORK in the makefile to enable that, and I don't know why that isn't used more widely) or by creating a helper process early on (before lots of memory is used) that will do the execs on your main process's behalf. Here's how to do that latter option:

# This is setup done at the start
set forkerProcess [open "|tclsh" r+]
fconfigure $forkerProcess -buffering line -blocking 0
puts $forkerProcess {
    fconfigure stdout -buffering none
    set tcl_prompt1 ""
    set tcl_prompt2 ""
    set tcl_interactive 0
    proc exechelper args {
        catch {exec {*}$args} value options
        puts [list [list $value $options]]
    }
}
# TRICKY BIT: Yield and drain anything unwanted
after 25
read $forkerProcess

# Call this, just like exec, to run programs without memory hazards
proc do-exec args {
    global forkerProcess
    fconfigure $forkerProcess -blocking 1
    puts $forkerProcess [list exechelper {*}$args]
    set result [gets $forkerProcess]
    fconfigure $forkerProcess -blocking 0
    while {![info complete $result]} {
        append result \n [read $forkerProcess]
    }
    lassign [lindex $result 0] value options
    return -options $options $value
}

Upvotes: 1

Related Questions