Itcl: inconsistency of $this variable

Question

While doing my project migration from Tcl 8.5.9/Itcl 3.4 to Tcl 8.6.6/Itcl 4.0.5 I've faced with inconsistency of $this variable depending how it's accessed. Here is the minimized testcase:

puts "Tcl  version : $tcl_patchLevel"
puts "Itcl version : [package require Itcl]"

itcl::class Base {
    public {
        method base_process {script} {
            uplevel $m_main main_process [list $script]
        }
        method set_main {main} {
          set m_main $main
        }
    }
    protected {
        variable m_main
    }
}

itcl::class Main {
  inherit Base
    public {
        method main_process {script} {
            uplevel $script
        }
    }
}

itcl::class Worker {
  inherit Base
    public {
        method worker_process_direct {} {
            puts "Direct query: this = $this"
        }
        method worker_process_inderect {} {
            base_process {puts "Indirect query: this = $this"}
        }
        method worker_process_both {} {
            puts "Direct query: this = $this"
            base_process {puts "Indirect query: this = $this"}
        }
    }
}

Main main

Worker worker
worker set_main main

puts "\n==== worker_process_direct   ===="
worker worker_process_direct

puts "\n==== worker_process_indirect ===="
worker worker_process_inderect

puts "\n==== worker_process_both     ===="
worker worker_process_both

worker_process_direct and worker_process_both functions always provide correct results. But worker_process_inderect works correctly only with old version of Tcl/Itcl. For Tcl 8.6.6/Itcl 4.0.5 $this variable is strangely changed to the instance of Main class instead of Worker.

Here is the output of the script above for two versions of Tcl/Itcl.

Tcl  version : 8.5.9
Itcl version : 3.4

==== worker_process_direct   ====
Direct query: this = ::worker

==== worker_process_indirect ====
Indirect query: this = ::worker          <<<<<<<<<<<< CORRECT

==== worker_process_both     ====
Direct query: this = ::worker
Indirect query: this = ::worker

Tcl  version : 8.6.6
Itcl version : 4.0.5

==== worker_process_direct   ====
Direct query: this = ::worker

==== worker_process_indirect ====
Indirect query: this = ::main            <<<<<<<<<< INCORRECT

==== worker_process_both     ====
Direct query: this = ::worker
Indirect query: this = ::worker

Did I miss something and there were significant changes in Tcl/Itcl which I haven't noticed?

Answer 1

Now that is very curious! I augmented your script to define Main like this:

itcl::class Main {
    inherit Base
    public {
        method main_process {script} {
            uplevel $script
            # Print what is actually going on!
            puts >>[tcl::unsupported::disassemble script $script]<<
        }
    }
}

With 8.5/3.4 I get this output:

Tcl  version : 8.5.9
Itcl version : 3.4

==== worker_process_direct   ====
Direct query: this = ::worker

==== worker_process_indirect ====
Indirect query: this = ::worker
>>ByteCode 0x0x7fedea044c10, refCt 1, epoch 3, interp 0x0x7fedea033010 (epoch 3)
  Source "puts \"Indirect query: this = $this\""
  Cmds 1, src 35, inst 12, litObjs 3, aux 0, stkDepth 3, code/src 0.00
  Commands 1:
      1: pc 0-10, src 0-34
  Command 1: "puts \"Indirect query: this = $this\""
    (0) push1 0     # "puts"
    (2) push1 1     # "Indirect query: this = "
    (4) push1 2     # "this"
    (6) loadScalarStk 
    (7) concat1 2 
    (9) invokeStk1 2 
    (11) done 
<<

==== worker_process_both     ====
Direct query: this = ::worker
Indirect query: this = ::worker
>>ByteCode 0x0x7fedea044c10, refCt 1, epoch 3, interp 0x0x7fedea033010 (epoch 3)
  Source "puts \"Indirect query: this = $this\""
  Cmds 1, src 35, inst 12, litObjs 3, aux 0, stkDepth 3, code/src 0.00
  Commands 1:
      1: pc 0-10, src 0-34
  Command 1: "puts \"Indirect query: this = $this\""
    (0) push1 0     # "puts"
    (2) push1 1     # "Indirect query: this = "
    (4) push1 2     # "this"
    (6) loadScalarStk 
    (7) concat1 2 
    (9) invokeStk1 2 
    (11) done 
<<

With 8.6/4.0 I get this instead:

Tcl  version : 8.6.3
Itcl version : 4.0.2

==== worker_process_direct   ====
Direct query: this = ::worker

==== worker_process_indirect ====
Indirect query: this = ::main
>>ByteCode 0x0x1009af010, refCt 1, epoch 136, interp 0x0x100829a10 (epoch 136)
  Source "puts \"Indirect query: this = $this"...
  Cmds 1, src 35, inst 12, litObjs 3, aux 0, stkDepth 3, code/src 0.00
  Commands 1:
      1: pc 0-10, src 0-34
  Command 1: "puts \"Indirect query: this = $this"...
    (0) push1 0     # "puts"
    (2) push1 1     # "Indirect query: this = "
    (4) push1 2     # "this"
    (6) loadStk 
    (7) strcat 2 
    (9) invokeStk1 2 
    (11) done 
<<

==== worker_process_both     ====
Direct query: this = ::worker
Indirect query: this = ::worker
>>ByteCode 0x0x1009b0210, refCt 1, epoch 136, interp 0x0x100829a10 (epoch 136)
  Source "puts \"Indirect query: this = $this"...
  Cmds 1, src 35, inst 11, litObjs 2, aux 0, stkDepth 3, code/src 0.00
  Commands 1:
      1: pc 0-9, src 0-34
  Command 1: "puts \"Indirect query: this = $this"...
    (0) push1 0     # "puts"
    (2) push1 1     # "Indirect query: this = "
    (4) loadScalar1 %v0 
    (6) strcat 2 
    (8) invokeStk1 2 
    (10) done 
<<

So, 8.5 uses the loadScalarStk instruction to read the variable in both (indirect) cases, whereas 8.6 uses loadStk and loadScalar1 to load the variable in the two cases. Which is mighty strange; I wouldn't expect loadScalar1 to appear in a script fragment (it needs a Local Variable Table) but at least it is picking up the expected value, whereas loadStk is just picking up the wrong value entirely. I've also tried using exactly the same value in the two places — with the script kept in a shared variable — but that produces the same output; it looks like in one place it is evaluating but picking up the wrong value (perhaps a variable resolver issue?) and in the other it is picking up the right value but for the wrong reasons (as the LVT shouldn't be used in a script fragment; that's for full procedures/methods only). Either way, it's Bad News.

Please file a bug report at http://core.tcl-lang.org/tcl/tktnew as this smells like several sorts of wrong behaviour compounded.