Macros, a form of metaprogramming,or more precisely Multi-stage programming, is a way to modify code as data during compile time. Thanks to this technique, programmers now have the power close to the compiler (although you can't change the syntax arbitrarily), which just also transforms the AST.

As said in Wikipedia, it can be considered as a form of partial evaluation that performs computations at compile-time.To make more analogies, it's also similar to the scenario for embedded DSL, where full access to the AST is given and any transformations are possible.

Mostly based on the official guide at https://docs.scala-lang.org/scala3/guides/macros/index.html , I will divide the topic into several parts and focus on theories than giving examples, while also referencing other languages:

• conversion between code and data(AST)

• operations on AST (AST for type or terms)

• get information from the child node of an AST

• organize code and modularity

Clarifying terminologies

• code: the runnable code as it is.

• data and AST: we treat the code as data structure. It's usually a tree so we call it AST(abstract syntax tree). Quoted code is a shorthand for manually written AST.

• conversion between code and data: usually called

  • quote : transform code to data

  • splice (or eval) : evaluate data as code

conversion between code and data

Dynamic languages like Lisp(quoted code) and javascript(tostring,eval) make it easy to toggle code and data. In Scala it's different since macros are also statically typed.

The data is called quoted code '{ ... } (like string "") and the process of converting data back to code is called splice ${ ... }(like ${} inside string). To insert AST(Expr[T]) into quoted code '{ ... }, we can use the syntax $expr or ${ expr }

import scala.quoted.* // imports Quotes, Expr

def inspectImpl(x: Expr[Any])(using Quotes): Expr[Any] =
  println(x.show)
  x
inline def inspect(inline x: Any): Any = ${ inspectImpl('x) }

inspectImpl goes from AST to AST as a normal Scala function, while inline def inspect has more magic, converting code into AST and invoking inspectImpl.

using Quotes contains extra info about Expr[_]. Depending on the desired manipulations for AST(called Expr[_] here), other APIs like using Type[T] can also be added like below:

def getTyped[T](x: Expr[T])(using Type[T], Quotes): Expr[T]

The Expr.value, Expr.valueOrAbort, and Expr.unapply methods turns the AST Expr to the value(code) if possible.

Idris Elaborator Reflection:

In Idris, a dependently typed language, the directive %runElab takes data and turn it into code. The monadic do block of type Elab a construct the AST:

idNat : Nat -> Nat
idNat = %runElab (do intro `{{x}}
                     fill (Var `{{x}})
                     solve)

for more details visit https://docs.idris-lang.org/en/latest/elaboratorReflection/elaborator-reflection.html#elaborator-reflection

Operations on AST

Expr[t] and Term

Expr[T] can be seen as wrappers around a Term for expressions, where T is the statically-known type of the term. Below, we use the extension method asTerm to transform an expression into a term

Type and TypeRepr

access type for term : t.Underlying

def evalAndUse[T](x: Expr[T])(using t: Type[T])(using Quotes) = '{
  val x2: t.Underlying = $x // eval AST Expr[T] into x2:T
}

Similarly, we can also see Type[T] as a wrapper over TypeRepr.

def f(x: Expr[Int])(using Quotes): Expr[Int] =
  import quotes.reflect.*
  val tree: Term = x.asTerm

def g[T: Type](using Quotes) =
  import quotes.reflect.*
  val tpe: TypeRepr = TypeRepr.of[T]

Both Terms and TypeReprs (and therefore Exprs and Types) have an associated symbol, the underlying untyped AST,exposing similar api as java's reflection:

• declaredFields and declaredMethods : fields ,members and methods inside a class def

• flags allows you to check multiple properties of a symbol

• companionClass and companionModule jump to and from the companion object/class

• TypeRepr.baseClasses list of symbols of classes extended by a type, like sealed trait.

• Symbol.pos the position where the symbol is defined, the source code of the definition, the filename where the symbol is defined,etc

get extra info from AST child node

sometimes we want to get the type of member of a class:

boxTpe.memberType(leafSym)

here we need the parent type boxTpe for a child symbol leafSym.

More details at https://docs.scala-lang.org/scala3/guides/macros/best-practices.html

Organize code

While working on the AST data structure, the heavy use of path dependent types for type safety adds extra complexity, so more care is needed when referring to types of the AST:

def getDecls(using Quotes)(rootType: quotes.reflect.TypeRepr) = {
    import quotes.reflect.*
    val typeSybs: Symbol = rootType.typeSymbol
    val decl: List[Symbol] = typeSybs.declarations
  }

quotes here is not an object, but a method with implicit parameter that can be used in import, which is a huge jump from Scala 2:

transparent inline def quotes(using inline q: Quotes): q.type = q

for keywords inline and transparent, check https://docs.scala-lang.org/scala3/guides/macros/inline.html

If you want to avoid this fancy inline transparent thing:

  def getDecls(using q: Quotes)(rootType: q.reflect.TypeRepr) = {}

conclusion

In this guide, I provide a more theoretical perspective compared to the official guide while covering less material. Although Scala 3 macros are no longer experimental, it's still a research area and thus under-documented. When combining the new dependent typing, the interaction is nontrivial. In addition, an open problem is: to what extent can we achieve full dependent types with macros?

I recommend the resources below while I'm also trying to understand more about macros :

https://github.com/nicolasstucki/

https://martinfriedrichberger.net/ecoop17.html