CODEWITHSUNDEEP
tcl
jel88
jel88

Reputation: 35

TCL API coverage : check if a TCL command have been called and tested exaustively in a test suite

Supposing I have a TCL API like :

namespaceXY::apiXY <value> -opt1 <value1> -opt2 <value2> -opt3 <value3>

This API is used (or maybe not) in a test suite (i.e thousands of tests).

How I can check if my API have been called + tested exhaustively (all options have been called/tested).

Many thanks

Upvotes: 1

Views: 184

Answers (2)

Donal Fellows
Donal Fellows

Reputation: 137567

See @SchelteBron's answer for covering commands.

Exhaustively testing all options is going to be tricky, since they could potentially all interact in complex ways and some may be mutually-exclusive (think about the standard Tcl lsearch command for example). However, auditing that all options are at least called in your own commands can be done by additional audit-only probes. Checking all the sensible combinations of them is a manual task; you probably need a coverage tool for that.

Auditing Options in C Commands

Assuming that you're dealing with the case where you've got a C command that uses Tcl_GetIndexFromObj() to parse the option name (this is common and recommended) and where you don't mind having a threading hazard (also pretty common) the idea is simple. Make an integer variable (probably with file scope) in your C code, bind it to a Tcl variable with Tcl_LinkVar(), then use the resulting index from your (successful) Tcl_GetIndexFromObj() call to set a bit in that integer variable that says that the option was parsed.

#ifdef AUDIT_OPTIONS
static int foobar_optionTracker;
#endif

    // in the implementation function, called FoobarImpl here for sake of argument
    int index;

    if (Tcl_GetIndexFromObj(interp, objPtr, optionNameTable, "option", 0, &index) != TCL_OK) {
        return TCL_ERROR;
    }
#ifdef AUDIT_OPTIONS
    foobar_optionTracker |= 1 << index;
    // Theoretically should call Tcl_UpdateLinkedVar() here, but for audit-only its not important
#endif
    switch (index) {
        // ...
    }

    // In your command registration function
    Tcl_CreateObjCommand(interp, "foobar", FoobarImpl, NULL, NULL);
#ifdef AUDIT_OPTIONS
    Tcl_LinkVar(interp, "optionTracker(foobar)", (void*) &foobar_optionTracker, TCL_LINK_INT);
#endif

With that in place, you can just read the array element optionTracker(foobar) from your Tcl test control code to see what options have been parsed (assuming you're happy with a bit-mask) in the foobar command since the last time the mask was reset. You reset the mask by just writing 0 to it.

Note that there's also Tcl_GetIndexFromObjStruct() in the C API, but auditing coverage of that is not significantly different from above.

Auditing Options in Tcl Commands

The equivalent of Tcl_GetIndexFromObj() in pure Tcl code is tcl::prefix match, but that doesn't return an index. Instead it returns the full option name that you can use with switch. Auditing that is most easily done with a full array. (This is morally the same as what the version for the C code does, but adapted to work with the optimal tools in a particular language.)

proc foobar {mandatoryArgument1 mandatoryArgument2 args} {
    # Parse other things here, set up the TABLE of option descriptors, etc.
    foreach option $args {
        set option [tcl::prefix match $TABLE $option]
        if {$::DoAudit} {
            set ::foobarAudit($option) 1
        }
        switch -- $option {
            # etc...
        }
    }

You can use things like array size foobarAudit to count the number of options actually used, or parray foobarAudit to print out what was actually used.

Upvotes: 0

Schelte Bron
Schelte Bron

Reputation: 4813

You can set an execution trace on the command. That way the signature of your command won't change. So you still get the same results if any code does info args namespaceXY::apiXY. Also error messages are not affected.

proc cmdtracer {cmd op} {
    global cmdtracer
    dict incr cmdtracer $cmd
}

trace add execution namespaceXY::apiXY enter cmdtracer

In the end you'll have a cmdtracer dict that contains the counts of each way the command was called. You will have to figure out yourself how to check if all options have been tested. There is not enough information in your question to provide suggestions for that part.

Upvotes: 3

Related Questions