Provide a macro that works like `.` but expands macros in its arguments first

[mlegls]

I'm currently using Hy with CadQuery, which is a Python library for parametric CAD that basically implements a DSL via chained method calls. An example from their Quickstart docs:

result = cq.Workplane("XY").box(height, width, thickness)\
      .faces(">Z").workplane().hole(diameter)\
      .faces(">Z").workplane() \
      .rect(height - padding,width - padding,forConstruction=True)\
      .vertices()\
      .cboreHole(2.4, 4.4, 2.1)

Currently, it seems like the hy "." special form overrides all cons forms inside of it to represent method calls, but in cases like this I think it would be really helpful to support macro expansion inside of the "." form. For example, I wrote the macro

(defmacro on [&rest place]
  `((~@place) (workplane)))

hoping that (on faces ">Z") would expand to (faces ">Z") (workplace) inside of a dot form (i.e. (. cq ...), but instead Hy assumes that "on" is the name of a method.

I'm sure there are other Python libraries implemented as chained method call DSLs, where allowing macros inside the dot form would be helpful. Namespace conflicts between macros and chained methods could be addressed by making explicitly quoted cons forms only refer to method calls.

Is there already a way in Hy to achieve what I'm trying to do, or are there any other issues with this proposal?

[Kodiologist]

Generally, Hy macros are expanded outside-in, not inside-out. This allows a macro to do whatever it likes with symbols, whether they happen to be macro names in the surrounding scope or not. Thus you can still do things like define a macro named cq that produces a (. …) form of the desired type.

[mlegls]

Ah, I see. Even so, I wonder if it would be useful then to either have a standard library version of . that expands inside out, or a macro that makes everything inside its scope expand inside out rather than outside in.

Sure I can define a cq macro where if the number of sub-macros I want to define is small, I just check first form against everything I want. Or if it becomes a larger DSL, I could implement something like a defcqmacro macro to extend it dynamically. But then importing existing macros into this scoped DSL becomes a whole problem, and I feel like there might be cases where having inside-out expansion for a form would just make things simpler.

[Kodiologist]

a macro that makes everything inside its scope expand inside out rather than outside in.

My intuition is that it would be very difficult to make this work correctly in nontrivial cases.

Sure I can define a cq macro where if the number of sub-macros I want to define is small, I just check first form against everything I want. Or if it becomes a larger DSL, I could implement something like a defcqmacro macro to extend it dynamically. But then importing existing macros into this scoped DSL becomes a whole problem…

I wouldn't recommend trying to implement your own macro system, but instead having cq produce forms that Hy can macroexpand. For example, if you arranged it such that

(cq
  (.Workplane "XY")
  (.box height width thickness)
  (on faces ">Z")
  (.hole diameter))

expands to

(.hole (on (.box (cq.Workplane "XY") height width thickness) faces ">Z") diameter)

then Hy can expand a macro (or call a function) named on in this result.

I feel like there might be cases where having inside-out expansion for a form would just make things simpler.

Perhaps you'd like hy.macroexpand, or Hyrule's macroexpand-all.

[mlegls]

Perhaps you'd like hy.macroexpand, or Hyrule's macroexpand-all.

Thanks for the suggestion. Is there a way in Hy to access all defined macros in the current scope as a set? My current approach allowing first-order expansions in . is:

(defn consp [x]
  (= (type x) hy.models.Expression))

(setv *defined-macros* #{'on})

(defmacro on [#* place]
  `((~@place) (workplane)))

(defmacro dsl [namespace #* rest]
  (setv result '())
  (for [form rest]
    (if (and (consp form) 
             (in (. form [0]) *defined-macros*))
      (+= result (hy.macroexpand form))
      (+= result `(~form))))
  `(. ~namespace ~@result))

but it would be nice if I could also make a version where *defined-macros* refers to all defined macros in scope.

I wouldn't recommend trying to implement your own macro system

What's the reasoning here? I think this kind of thing is a pretty typical idiom in Common Lisp, at least. It would basically be extending this snippet so that DSLs can each have their own *defined-macros* special parameter, and then a few macros to manage the parameter. It would be more complicated if we wanted the macros be inaccessible from outside the DSL scope, but I don't think that's especially necessary.

[Kodiologist]

Is there a way in Hy to access all defined macros in the current scope as a set?

No, macro introspection in its full generality is not implemented.

I wouldn't recommend trying to implement your own macro system

What's the reasoning here?

Because it's a lot more work than just using the existing macro system.

[mlegls]

Because it's a lot more work than just using the existing macro system.

Right...

I wouldn't recommend trying to implement your own macro system, but instead having cq produce forms that Hy can macroexpand. For example, if you arranged it such that...

So this is just the thread first macro after all, and for example we could have

(defmacro on [body #* place]
  `(-> ~body (~@place) (.workplane)))

(-> cq 
     (.Workplane "XY")
     (.box height width thickness)
     (on .faces ">Z") (.hole diameter))

but this requires defining all the inner macros in terms of the -> expansion, which I imagine gets pretty complicated compared with only having to think about what the direct expansion looks like within the macro it's a part of.

No, macro introspection in its full generality is not implemented.

Is this as in not implemented yet, or is it against the language design philosophy?

[Kodiologist]

but this requires defining all the inner macros in terms of the -> expansion

Right, probably better to define on as (.workplane (~@place) it) instead.

No, macro introspection in its full generality is not implemented.

Is this as in not implemented yet, or is it against the language design philosophy?

As in "it would be nice to have, but nobody's working on it at the moment or has plans to work on it".

[Kodiologist]

Anyway, let's narrow this issue to a single feature request so we can decide what to do with it. You've brought up a few things. What would you like to request? Is it what you said in the first post, for (. …) to expand macros itself instead of following the usual rules?

[mlegls]

Yeah, either that or have an alternate version of . that does so.

[Kodiologist]

I think the second is more likely to happen, and Hyrule's the place for that, so I'll move this issue over.

[mlegls]

Here's my partial proposed implementation, which makes use of the existing . and macroexpand-all:

(defn consp [x]
  (= (type x) hy.models.Expression))

(defn dotformp [x]
  (in (type x) #{hy.models.Expression hy.models.List}))

  (defmacro @ [namespace #* rest]
    (setv result `(. ~namespace))
    (for [form rest]
      (if (consp form)
        (let [expanded (hyrule.macroexpand-all form)]
         (+= result (if (dotformp (. expanded [0]))
                      expanded
                      `(~expanded))))
        (+= result `(~form))))
    result)

Not sure if what I'm calling consp and dotformp already have more canonical definitions.

The problem is, this implementation doesn't support macro expansions that start with an a symbol, which is allowable in . forms. I tried

(defn dotformp [x [module None]]
  (or (in (type x) #{hy.models.Expression hy.models.List})i
      (in (str x) (dir module))))

(defmacro @ [namespace #* rest]
  (setv result `(. ~namespace))
  (for [form rest]
    (if (consp form)
      (let [expanded (hyrule.macroexpand-all form)
            to-add   (if (dotformp (. expanded [0]) 
                                   result)
                        expanded
                        `(~expanded))]
       (+= result to-add))
      (+= result `(~form))))
  result)

but couldn't figure out how to make result compile during macro execution to be suitable as the module arg of dotformp. Is what I'm trying to do supported? Does the built-in . even do this dir check during compile time?

Maybe there's another way to distinguish between when the macro expansion starts with a symbol because macroexpand-all tried to expand a function vs because the macro output starts with an object property, but I can't figure how without deeper macro introspection capabilities...

[mlegls]

Oh wait I'm silly. I thought it was some spooky Python error but there's just an extra i in my dotformp definition because I use vim lmao. I think the second implementation just works.