When You Get to Be Smart Writing a Macro

Consider a macro `(replace-here replacement expression)` that replaces every instance of `here` with `replacement` in `expression`:

    (replace-here 10
      (* here (+ 5 here))

    (* 10 (+ 5 10))

    (defn foo [] here)

    (replace-here 10
      (foo))

We can see this I practice in core.async where the `(go expr)` macro will execute `expr` asynchronously and block (await) when it encounters a `(<! channel)` call.

BUT `<!` must be visible to the go macro, because the macro transforms the code into a state machine that can suspend and resume when the call blocks and unblocks. Than means that this code will await the channel:

    (async/go
      (let [val (async/<! ch)]
        (println “got” val)))

    (defn foo [ch]
       (async/<! ch))

    (async/go
      (let [val (foo ch)]
        (println “got” val)))

Practically, this means you cannot create helper functions that operate on channels, you must always wrap the channel operation in a go block so that the macro will see it.

In practice it’s not that bad, you learn to structure your code into ways that avoid it. But it’s a limitation nonetheless and one that only exists because core.async is implemented as macros.

As an aside, another problem I have with core.async that exists because it’s a macro-based implementation rather than a first class citizen is that there are occasions where the code transformations mean that an error can occur somewhere that isn’t in your source code. I have had a situation where there was an error and the stack trace was entirely in Clojure’s code without referencing my code in any way. Imagine how difficult it is to debug when you don’t even know what source file of yours contains the code with the error! If it was built into the language rather than as macros, then it could at least retain source location contextual data to be passed to whatever internal code raised the error. But that’s a separate issue from the “see inside” issue.

For instance, you can't put the "break" of a "for" loop in a helper function called out of the for loop. It has to be enclosed in the for loop.

We can think of the loop as a macro; Lisp would implement it as such.

When macros transform certain expressions enclosed in the macro call, all the syntax has to be right there, enclosed in the call.

In other words macros are "local syntactic transformations". Why I'm putting that in quotes is that this is the exact phrase used in the paper "Macros that Reach Out and Touch Somewhere" by George Kiczales et al.

Kiczales describes a macro system which peforms global program transformations, allowing macros to act on code that is not enclosed in them; i.e. bring about nonlocal transformations.

"In this paper, we present a new kind of macro, called a data path macro, in which transformations can take place at any point along the dataflow path that includes the macro invocation."

This is pretty exotic. I think nbody has done anything like it, and they never released their code. They left unsolved problems documented in section 5, Future Work.