Can runtime information be used during macro expansion in racket?

Question

Say I have a hash table at runtime that has strings as keys. Can a macro have access to this information and build a let expression from it?

(define env (hash 'a 123 'b 321))

(magic-let env (+ a b)) ; 444

I know I can hack around with with identifier-binding by replacing non-defined identifiers with a lookup in the hash table but then shadowing will not work as in a normal let.

Tagging scheme too as I assume its macro system is similar.

Answer 1

No, you can’t do that. At least not the way you describe.

The general reason why you cannot access runtime values within macros is simple: macros are fully expanded at compile time. When your program is compiled, the runtime values simply do not exist. A program can be compiled, and the bytecode can be placed on another computer, which will run it weeks later. Macro-expansion has already happened. No matter what happens at runtime, the program isn’t going to change.

This guarantee turns out to be incredibly important for a multitude of reasons, but that’s too general a discussion for this question. It would be relevant to discuss a particular question, which is why bindings themselves need to be static.

In Racket, as long as you are within a module (i.e. not at the top-level/REPL), all bindings can be resolved statically, at compile-time. This is a very useful property in other programming languages, mostly because the compiler can generate much more efficiently optimized code, but it is especially important in Racket or Scheme. This is because of how the macro system operates: in a language with hygienic macros, scope is complicated.

This is actually a very good thing—it is robust enough to support very complex systems that would be much harder to manage without hygiene—but it introduces some constraints:

1. Since every binding can be a macro or a runtime value, the binding needs to be known ahead of time in order to perform program expansion. The compiler needs to know if it needs to perform macro expansion or simply emit a variable reference.

2. Additionally, scoping rules are much more intricate because macro-introduced bindings live in their own scope. Because of this, binding scopes do not need to be strictly lexical.

Your magic-let could not work quite as you describe because the compiler could not possibly deduce the bindings for a and b statically. However, all is not lost: you could hook into #%top, a magical identifier introduced by the expander when encountering an unbound identifier. You could use this to replace unbound values with a hash lookup, and you could use syntax parameters to adjust #%top hygienically within each magic-let. Here’s an example:

#lang racket

(require (rename-in racket/base [#%top base-#%top])
         racket/stxparam)

(define-syntax-parameter #%top (make-rename-transformer #'base-#%top))

(define-syntax-rule (magic-let env-expr expr ...)
  (let ([env env-expr])
    (syntax-parameterize ([#%top (syntax-rules ()
                                   [(_ . id) (hash-ref env 'id)])])
      (let () expr ...))))

(magic-let (hash 'a 123 'b 321) (+ a b)) ; => 444

Of course, keep in mind that this would replace all unbound identifiers with hash lookups. The effects of this are twofold. First of all, it will not shadow identifiers that are already bound:

(let ([a 1])
  (magic-let (hash 'a 2)
    a)) ; => 1

This is probably for the best, just to keep things semi-sane. It also means that the following would raise a runtime exception, not a compile-time error:

(magic-let (hash 'a 123) (+ a b))
; hash-ref: no value found for key
;   key: 'b

I wouldn’t recommend doing this, as it goes against a lot of the Racket philosophy, and it would likely cause some hard-to-find bugs. There’s probably a better way to solve your problem without abusing things like #%top. Still, it is possible, if you really want it.