Common Lisp: How do I set a variable in my parent's lexical scope?

Question

I want to define a function (not a macro) that can set a variable in the scope its called.

I have tried:

(defun var-set (var-str val)
  (let ((var-interned
          (intern (string-upcase var-str))))
    (set var-interned val)
    ))

(let ((year "1400"))
  (var-set "year" 1388)
  (labeled identity year))

Which doesn't work because of the scoping rules. Any "hacks" to accomplish this?

In python, I can use

previous_frame = sys._getframe(1)
previous_frame_locals = previous_frame.f_locals
previous_frame_locals['my-var'] = some_value

Any equivalent API for lisp?

Answer 1

That Python mechanism violates the encapsulation which motivates the existence of lexical scope.

Because a lexical scope is inaccessible by any external means other than invocations of function bodies which are in that scope, a compiler is free to translate a lexical scope into any representation which performs the same semantics. Variables named in the source code of the lexical scope can disappear entirely. For instance, in your example, all references to year can be replaced by copies of the pointer to the "1400" string literal object.

Separately from the encapsulation issue there is also the consideration that a function does not have any access at all to a parent lexical scope, regardless of that scope's representation. It does not exist. Functions do not implicitly pass their lexical scope to children. Your caller may not have a lexical environment at all, and there is no way to know. The essence of the lexical environment is that no aspect of it is passed down to children, other than via the explicit passage of lexical closures.

Python's feature is poorly considered because it makes programs dependent on the representation of scopes. If a compiler like PyPy is to make that code work, it has to constrain its treatment of lexical scopes to mimic the byte code interpreted version of Python.

Firstly, each function has to know who called it, so it has to receive some parameter(s) about that, including a link to the caller's environment. That's going to be a source of inefficiency even in code that doesn't take advantage of it.

The concept of a well-defined "previous frame" means that the compiler cannot merge together frames. Code which expects some variable to be in the third frame up from here will break if those frames are all inlined together due to a nested lexical scope being flattened, or due to function inlining.

As soon as you provide an interface to the parent lexical environment, and applications start using it, you no longer have lexical scoping. You have a form of dynamic scoping with lexical-like visibility rules.

The application logic can implement de facto dynamic scope on top of this API, because you can write a loop which searches for a variable across the chain of lexical scopes. Does my parent have an x variable? If not, does the grandparent, if there is one? You can search the dynamic chain of invocations for the most recent one which binds x, and that is dynamic scope.

There is nothing wrong with dynamic scope, if it is a separate discipline that is not entangled in the implementation of lexical scope.

That said, an API for tracing frames and getting at local variables is is the sort of introspection that is very useful in developing a debugger. Another angle on this is that if you work that API into an application, you're using debugging features in production.

Answer 2

You can't. This is why it is called lexical scope: you have access to variable bindings if and only if you can see them. The only way to get at such a binding is to create some object for which it is visible and use that. For instance:

(defun foo (x)
  (bar (lambda (&optional (v nil vp)
          (if vp (setf x vp) x))))

(defun bar (a)
  ...
  (funcall a ...))

Some languages, such as Python have both rather rudimentary implementations of variable bindings and a single implementation (or a mandated implementation) which allow you to essentially poke around inside the system to subvert lexical scoping. Some CL implementations may have rudimentary implementation of variable bindings (probably none do) but Common Lisp the language does not mandate such implementations and nor should it.

As an example of the terrible consequences of mandating that some kind of access to lexical variables must be allowed consider this:

(defun outer (n f)
  (if (> n 0)
      (outer (g n) f)
      (funcall f)))

If f could somehow poke at the lexical bindings of outer this would mean that all those bindings would need to exist at the point f was called: tail-call elimination would thus be impossible. If the language mandated that such bindings should be accessible then the language is mandating that tail-call elimination is not possible. That would be bad.

(It is quite possible that implementations, possibly with some debugging declarations, allow such access in some circumstances. That's very different than the language mandating such a thing.)

Answer 3

What are you trying to achieve?

What about a closure? Write the defun inside the let, and create an accessor and a setter function if needed.

(let ((year "1400"))
  (defun current-year ()
    year)

  (defun set-year (val)
    (setf year val)))

and

CL-USER> (current-year)
"1400"
CL-USER> (set-year 1200)
1200
CL-USER> (current-year)
1200

Answer 4

(defvar *lexical-variables* '())

(defun get-var (name)
  (let ((var (cdr (assoc name *lexical-variables*))))
    (unless var (error "No lexical variable named ~S" name))
    var))

(defun deref (var)
  (funcall (if (symbolp var)
               (or (cdr (assoc var *lexical-variables*))
                   (error "No lexical variable named ~S" var))
               var)))

(defun (setf deref) (new-value var)
  (funcall (if (symbolp var)
               (or (cdr (assoc var *lexical-variables*))
                   (error "No lexical variable named ~S" var))
               var)
           new-value))

(defmacro with-named-lexical-variable ((&rest vars) &body body)
  (let ((vvar (gensym))
        (vnew-value (gensym))
        (vsetp (gensym)))
    `(let ((*lexical-variables* (list* ,@(mapcar (lambda (var)
                                                   `(cons ',var
                                                          (lambda (&optional (,vnew-value nil ,vsetp))
                                                            (if ,vsetp
                                                                (setf ,var ,vnew-value)
                                                                ,var))))
                                                 vars)
                                       *lexical-variables*)))
       ,@body)))


(defun var-set (var-str val)
  (let ((var-interned (intern (string-upcase var-str))))
    (setf (deref var-interned) val)))


(let ((x 1)
      (y 2))
  (with-named-lexical-variable (x y)
    (var-set "x" 3)
    (setf (deref 'y) 4)
    (mapcar (function deref) '(x y))))
;; -> (3 4)


(let ((year "1400"))
  (with-named-lexical-variable (year)
    (var-set "year" 1388))
  year)
;; --> 1388

Answer 5

You cannot do that because after compilation the variable might not even exist in any meaningful sense. E.g., try to figure out by looking at the output of (disassemble (lambda (x) (+ x 4))) where you would write the new values of x.

You have to tell both the caller and the callee (at compile time!) that the variable is special:

(defun set-x (v)
  (declare (special x))
  (setq x v))
(defun test-set (a)
  (let ((x a))
    (declare (special x))
    (set-x 10)
    x))
(test-set 3)
==> 10

See Dynamic and Lexical variables in Common Lisp for further details on lexical vs dynamic bindings.