Macro file is required multiple times during compilation

[datwaft]

Hello!

This issue was kind of a pain to debug.

One of the functionalities that I make use of in my macros to configure Neovim is that a global inside a macro file is defined when the macro file is required and then can be used by any macro during the compilation.

An example of a use of this is a global counter:

(set _G.__core_counter 0)

This counter can be used in the macros inside the file to generate an unique symbol each time without needing to use (os.date %s) or the like.

This functionality is present in the normal Fennel behaviour as can be seen in the reproduction repo.

In hotpot the macro file is evaluated each time it is required, which by itself goes against the normal behaviour of Fennel, but it seems that the counter is reset in each evaluation of the macro file, as if it wasn't set.

How to reproduce the issue

I created datwaft/hotpot-issue_38 to reproduce the issue using Docker.

Expected results

This can be seen by executing the following commands inside the Docker container:

foo@bar:~$ cd test && fennel init.fnl
The macros.fnl file was required.
The file1.fnl file was required.
file1   1
file1   2
file1   3
The file2.fnl file was required.
file2   4
file2   5
file2   6
The file3.fnl file was required.
file3   7
file3   8
file3   9

As you can see the macros.fnl file is only required once and the counter is maintained between files.

Current results

The macros.fnl file was required.
The file1.fnl file was required.
file1 1
file1 2
file1 3
The macros.fnl file was required.
The file2.fnl file was required.
file2 1
file2 2
file2 3
The macros.fnl file was required.
The file3.fnl file was required.
file3 1
file3 2
file3 3

As you can see the macros.fnl file is required once per file and the counter is not maintained.

[rktjmp]

This is by design f9a4c29 so hotpot can track if a module needs recompilation due to a macro file changing. If the macro is only loaded once, only the first file to use it is able to "mark" the macro file as a dependency because all other requires won't hit the searcher.

I have been meaning to look into adding a compiler event to Fennel itself that would let hotpot get notified when a macro is required (maybe? via a compiler plugin) but haven't had the time to look or discuss if that patch would be accepted or check if it would even solve the issue.

I agree that this does break the "just fennel" rule, but it's needed to make hotpot work otherwise you'd have to manually compile/touch anything that required a macro after altering said macro, at which point you might as well just be running fennel -c directly!

Can you get around your issue by adding a "has-init'd" guard?

(if (not _G.__core_counter)
  (set _G.__core_counter 0))

---

Also care, the counter may not behave like you expect since Hotpots code is compiled as needed when something is changed, not in one monolithic set as when you test it in Fennel.

edit keymap.fnl, edit settings.fnl, requires macro.fnl -> _G.counter = 0
hotpot compiles
  keymap.my_mac_0 = fn ...
  keymap.my_mac_1 = fn ...
  settings.my_mac_2 = fn ... (I know this would be 0 due to the OP issue but for examples sake, imagine there is a "don't reset to 0" guard)

# time passes
edit settings.fnl, requires macro.fnl -> _G.counter = 0
hotpot compiles
  settings.my_mac_0 = fn ...

I had the same issues with #6 using gensym which basically does what you're doing probably (autoincrementing names) but because of the cross/inter compile cycle I had to use a timestamp, though there are probably more elegant ways. Since it's only run at compile time and you're already throwing speed out you could probably just checksum the incoming list or something for a real uid.

So in real world usage of Hotpot, assuming you don't nuke the cache every time you make a change, it's more like this, with multiple compile passes:

root@73540473deb0:~/test# cat init2.fnl
; simulate a separate "partial" compile by hotpot where only file3 was changed
(require :file3)

# pretend this is hotpot booting with no cache
root@73540473deb0:~/test# fennel init.fnl
The macros.fnl file was required.
The file1.fnl file was required.
file1	1
file1	2
file1	3
The file2.fnl file was required.
file2	4
file2	5
file2	6
The file3.fnl file was required.
file3	7
file3	8
file3	9

# pretend this is hotpot booting with a cache, but you edited file3 meaning file3.lua is "stale"
root@73540473deb0:~/test# fennel init2.fnl
The macros.fnl file was required.
The file3.fnl file was required.
file3	1 <== clobbered
file3	2 <== clobbered
file3	3 <== clobbered

Annoying, I know!

[bangedorrunt]

@rktjmp is my problem related?

My command! macro:

(fn fn? [obj]
  "Returns true if the object is a function
  This only works at compilation time"
  (and (list? obj) (or (= (->str (first obj)) :hashfn)
                       (= (->str (first obj)) :fn))))

(fn pug [val prefix?]
  (let [inter-compile-uid (_G.os.date "%s")
        name (if prefix?
                  (.. (->str (gensym prefix?)) inter-compile-uid)
                  (.. (->str (gensym :pug)) inter-compile-uid))]
  `(do
     (tset _G ,name ,val)
     ,name)))

(fn vlua [what prefix?]
  `(.. "v:lua." ,(pug what prefix?) "()"))

;; BUG: `vlua` seems to have side-effect
(fn command! [name expr]
  (let [name (->str name)]
     (if (fn? expr)
       `(vim.cmd (string.format "command! %s call %s" ,name ,(vlua expr)))
       `(vim.cmd (string.format "command! %s %s" ,name ,expr)))))

Open LspLog command:

(fn open-lsp-log []
  (let [path (vim.lsp.get_log_path)]
    (vim.cmd (.. "edit " path))))

;; with `vlua`: SIDE-EFFECTS
(command! :LspLog #(open-lsp-log))
;; without `vlua`: WORK
;; (command! :LspLog "call v:lua.tdt.lsp.open_lsp_log()")

Actual behaviors: file paths are spammed multiple times when opening

Screen_Recording_2021-09-25_at_3.12.28_pm.mov

Expected behavior: nvim open file directly without path printed on messages

[rktjmp]

I don't think so but it's a bit hard to tell without a repro. Maybe open a separate issue and post the complied lua too.

[bangedorrunt]

@rktjmp my bad, just a minute ago, I suddenly realized I was using aniseed, not hotpot 🤣

[datwaft]

This is by design f9a4c29 so hotpot can track if a module needs recompilation due to a macro file changing. If the macro is only loaded once, only the first file to use it is able to "mark" the macro file as a dependency because all other requires won't hit the searcher.

Ah, what a pain.

Can you get around your issue by adding a "has-init'd" guard?

Sadly, no.

It was one of the first workarounds I tested, the if is always true because the value is not saved between imports of the macro file.

I had the same issues with #6 using gensym which basically does what you're doing probably (autoincrementing names) but because of the cross/inter compile cycle I had to use a timestamp, though there are probably more elegant ways. Since it's only run at compile time and you're already throwing speed out you could probably just checksum the incoming list or something for a real uid.

One of the workarounds I tried was using that timestamp trick, but, because Lua only uses seconds as the timestamp, the identifiers are still repeated.

The workaround that I am currently using is generating a uuid as an identifier, seeding the randomness with the current timestamp.

It still doesn't work sometimes, so I think I will have to generate a checksum based on the arguments or something.

Sadly this bug breaks my macros related to mappings, autocmds, commands, or anything that requires a generated identifier for a global Lua variable. What a pain.

Can't you create an option for just "running fennel -c" on compilation? I don't really care if the compilation is slow, only that it behaves as expected.

🥲

[rktjmp]

It was one of the first workarounds I tested, the if is always true because the value is not saved between imports of the macro file.

Hmm, that sounds like part of Fennels sandboxing. Hotpot wont intentionally unset anything but I would bet the _G you're setting in the macro isn't actually making it out to the "real" global namespace.

Actually I bet it's a bug in Fennel that hasn't made a release yet, which actually popped up because I was running into a related issue bakpakin/Fennel#381, bakpakin/Fennel@e461e24

Disable compiler env sandbox and add the guard:

diff --git a/config/fnl/core/macros.fnl b/config/fnl/core/macros.fnl
index 7cb1171..282fcc2 100644
--- a/config/fnl/core/macros.fnl
+++ b/config/fnl/core/macros.fnl
@@ -6,7 +6,9 @@
 ;; module.
 (print "The macros.fnl file was required.")

-(set _G.__core_counter 0)
+(print _G.__core_counter)
+(if (not _G.__core_counter)
+  (set _G.__core_counter 0))

 (fn display-counter [name]
   "This is a small macro that displays the counter current value."
diff --git a/config/init.lua b/config/init.lua
index 55ae790..866ac7e 100644
--- a/config/init.lua
+++ b/config/init.lua
@@ -7,5 +7,13 @@ if vim.fn.empty(vim.fn.glob(hotpot_path)) > 0 then
   })
 end

-require("hotpot").setup()
+require("hotpot").setup({
+  compiler = {
+    macros = {
+      env = "_COMPILER",
+      compilerEnv = _G,
+      allowedGlobals = false,
+    }
+  }
+})
 require("core")

Replace ~/.local/share/nvim/site/pack/packer/start/hotpot.nvim/lua/hotpot/fennel.lua with HEAD copy of Fennel (with the same patch as f9a4c29 applied).

You have to do this because the latest stable fennel release doesn't actually pass the compiler options correctly.

Profit.

Can't you create an option for just "running fennel -c" on compilation? I don't really care if the compilation is slow, only that it behaves as expected.

No not really... That's basically just disabling hotpot.

[rktjmp]

For what it's worth, I hit similar pains with macros and how Fennel expects you to behave when using them.

Hotpot will only track the latest stable, but because of this, it will only change that file very rarely.

If you fork Hotpot and do your HEAD patch to lua/hotpot/fennel.lua, you should be able to rebase on master pretty regularly without any kind of merge issues.

I would like to not need the patch, but yeah, haven't gotten to trying to upstream a solution yet.

[rktjmp]

bakpakin/Fennel#398

[datwaft]

At the end I followed your idea and implemented my unique symbol generator using checksum, specifically kikito/md5.lua.

Here is the implementation, it works like a charm (but it's not foolproof, if two function definitions are identical it can break):

(macro gensym [...]
  (local {:sumhexa md5} (require :md5))
  (->> (icollect [_ expr (ipairs [...])]
                 (tostring expr))
       (string.concat)
       (md5)
       (string.format "__%s")
       (sym)))

Ideally I would have used fennel.view instead of tostring to convert the expression to a string, but if I try to do:

(local {: view} (require :fennel))

It says that view is nil.

Can you test it on your side? This only happens when inside a macro.

I can open an issue if you need to, with a reproduction using Docker.

[rktjmp]

(but it's not foolproof, if two function definitions are identical it can break)

Isn't that the idea behind hashing them? 😄

Not sure if that's an edited paste or not, but you might want to use a different name since gensym already exists in the Fennel macro env.

Seems to works fine for me, have you got provide_require_fennel = true enabled? What is (require :fennel) returning?

[datwaft]

Isn't that the idea behind hashing them? 😄

Yes 😆, but if the functions access different values with the same symbol, they would be considered equal, which would create problems.

e.g.

;;; File1
(let [x 10] (gensym #(+ x 10)))
;;; File 2
(let [x 20] (gensym #(+ x 10)))

Not sure if that's an edited paste or not, but you might want to use a different name since gensym already exists in the Fennel macro env.

I am using core/gensym in my configuration, just omitted the first part when copying it here.

Seems to works fine for me, have you got provide_require_fennel = true enabled?

Yes.

What is (require :fennel) returning?

I cannot reproduce it at this moment, it seems to be irregular behaviour, when I get to reproduce it again I will open an issue.

[datwaft]

I could reproduce it on my Neovim configuration but not on Docker.

When using it on my vim configuration macro:

Click to see the result of (require :fennel)

      ["fnl/arglist"] = { "([f col])", "([f val col])" },
      ["fnl/docstring"] = "Reduce collection `col' using function `f' and optional initial value `val'.\n\n`f' should be a function of 2 arguments.  If val is not supplied,\nreturns the result of applying f to the first 2 items in coll, then\napplying f to that result and the 3rd item, etc.  If coll contains no\nitems, f must accept no arguments as well, and reduce returns the\nresult of calling f with no arguments.  If coll has only 1 item, it is\nreturned and f is not called.  If val is supplied, returns the result\nof applying f to val and the first item in coll, then applying f to\nthat result and the 2nd item, etc.  If coll contains no items, returns\nval and f is not called.  Calls `seq' on `col'.\n\nEarly termination is possible with the use of `reduced'\nfunction.\n\n# Examples\nReduce sequence of numbers with `add'\n\n``` fennel\n(reduce add [1 2 3 4])\n;; => 10\n(reduce add 10 [1 2 3 4])\n;; => 20\n```"
    },
    [<function 197>] = {
      ["fnl/arglist"] = { "[col]" },
      ["fnl/docstring"] = "Returns everything but the last element of a table as a new\n  table. Calls `seq' on its argument."
    },
    [<function 198>] = {
      ["fnl/arglist"] = { "([])", "([x])", "([x y])", "([x y & xs])" },
      ["fnl/docstring"] = "Concatenate tables."
    },
    [<function 199>] = {
      ["fnl/arglist"] = { "[binding test]", "then-branch", "else-branch" }
    },
    [<function 200>] = {
      ["fnl/arglist"] = { "[x]" },
      ["fnl/docstring"] = "If `x' is empty, returns `nil', otherwise `x'."
    },
    [<function 201>] = {
      ["fnl/arglist"] = { "[x]" },
      ["fnl/docstring"] = "Test if `x' is `true'"
    },
    [<function 202>] = {
      ["fnl/arglist"] = { "tbl", "key1", "..." },
      ["fnl/docstring"] = "Look up key1 in tbl table. If more args are provided, do a nested lookup."
    },
    [<function 203>] = {
      ["fnl/arglist"] = { "([])", "([a])", "([a b])", "([a b c])", "([a b c d])", "([a b c d & rest])" },
      ["fnl/docstring"] = "Sum arbitrary amount of numbers."
    },
    [<function 204>] = {
      ["fnl/arglist"] = { "[x]" },
      ["fnl/docstring"] = "Test if `x' is a Boolean"
    },
    [<function 205>] = {
      ["fnl/arglist"] = { "([x])", "([x y])", "([x y & xs])" },
      ["fnl/docstring"] = "Deep compare values.\n\n# Examples\n\n`eq' can compare both primitive types, tables, and user defined types\nthat have `__eq` metamethod.\n\n``` fennel\n(assert-is (eq 42 42))\n(assert-is (eq [1 2 3] [1 2 3]))\n(assert-is (eq (hash-set :a :b :c) (hash-set :a :b :c)))\n(assert-is (eq (hash-set :a :b :c) (ordered-set :c :b :a)))\n```\n\nDeep comparison is used for tables which use tables as keys:\n\n``` fennel\n(assert-is (eq {[1 2 3] {:a [1 2 3]} {:a 1} {:b 2}}\n               {{:a 1} {:b 2} [1 2 3] {:a [1 2 3]}}))\n(assert-is (eq {{{:a 1} {:b 1}} {{:c 3} {:d 4}} [[1] [2 [3]]] {:a 2}}\n               {[[1] [2 [3]]] {:a 2} {{:a 1} {:b 1}} {{:c 3} {:d 4}}}))\n```"
    },
    [<function 206>] = {
      ["fnl/arglist"] = { "a", "b", "..." },
      ["fnl/docstring"] = "Arithmetic operator; works the same as Lua but accepts more arguments."
    },
    [<function 207>] = {
      ["fnl/arglist"] = { "[x]" },
      ["fnl/docstring"] = "Test if `x' is a number without floating point data.\n\nNumber is rounded with `math.floor' and compared with original number."
    },
    [<function 208>] = {
      ["fnl/arglist"] = { "[x]" },
      ["fnl/docstring"] = "Test if `x' is a number with floating point data."
    },
    [<function 209>] = {
      ["fnl/arglist"] = { "[&", "args]" },
      ["fnl/docstring"] = "Constructs sequential table out of it's arguments.\n\nSets additional metadata for function `vector?' to work.\n\n# Examples\n\n``` fennel\n(local v (vector 1 2 3 4))\n(assert-eq v [1 2 3 4])\n```"
    },
    [<function 210>] = {
      ["fnl/arglist"] = { "[x]" },
      ["fnl/docstring"] = "Test if `x' is a positive integer."
    },
    [<function 211>] = {
      ["fnl/arglist"] = { "[x]" },
      ["fnl/docstring"] = "Test if `x' is a Boolean"
    },
    [<function 212>] = {
      ["fnl/arglist"] = { "\n  ([f col])", "\n  ([f col1 col2])", "\n  ([f col1 col2 col3])", "\n  ([f col1 col2 col3 & cols])" },
      ["fnl/docstring"] = "Maps function `f' over one or more collections.\n\nAccepts arbitrary amount of collections, calls `seq' on each of it.\nFunction `f' must take the same amount of arguments as the amount of\ntables, passed to `mapv'. Applies `f' over first value of each\ntable. Then applies `f' to second value of each table. Continues until\nany of the tables is exhausted. All remaining values are\nignored. Returns a sequential table of results.\n\n# Examples\nMap `string.upcase' over the string:\n\n``` fennel\n(mapv string.upper \"string\")\n;; => [\"S\" \"T\" \"R\" \"I\" \"N\" \"G\"]\n```\n\nMap `mul' over two tables:\n\n``` fennel\n(mapv mul [1 2 3 4] [1 0 -1])\n;; => [1 0 -3]\n```\n\nBasic `zipmap' implementation:\n\n``` fennel\n(import-macros {: into} :cljlib.init-macros)\n(fn zipmap [keys vals]\n  (into {} (mapv vector keys vals)))\n\n(zipmap [:a :b :c] [1 2 3 4])\n;; => {:a 1 :b 2 :c 3}\n```"
    },
    [<function 213>] = {
      ["fnl/arglist"] = { "([])", "([a])", "([a b])", "([a b c])", "([a b c d])", "([a b c d & rest])" },
      ["fnl/docstring"] = "Sum arbitrary amount of numbers."
    },
    [<function 214>] = {
      ["fnl/arglist"] = { "tbl", "method-name", "..." },
      ["fnl/docstring"] = "Call the named method on tbl with the provided args.\nMethod name doesn't have to be known at compile-time; if it is, use\n(tbl:method-name ...) instead."
    },
    [<function 215>] = {
      ["fnl/arglist"] = { "[pred", "tbl]" },
      ["fnl/docstring"] = "Test if every item in `tbl' satisfies the `pred'."
    },
    [<function 216>] = {
      ["fnl/arglist"] = { "[col]" },
      ["fnl/docstring"] = "Create sequential table.\n\nTransforms original table to sequential table of key value pairs\nstored as sequential tables in linear time.  If `col' is an\nassociative table, returns sequential table of vectors with key and\nvalue.  If `col' is sequential table, returns its shallow copy.  If\n`col' is string, return sequential table of its codepoints.\n\n# Examples\nSequential tables remain as is:\n\n``` fennel\n(seq [1 2 3 4])\n;; [1 2 3 4]\n```\n\nAssociative tables are transformed to format like this `[[key1 value1]\n... [keyN valueN]]` and order is non deterministic:\n\n``` fennel\n(seq {:a 1 :b 2 :c 3})\n;; [[:b 2] [:a 1] [:c 3]]\n```\n\nSee `into' macros for transforming this back to associative table.\nAdditionally you can use `conj' and `apply' with\n`hash-map':\n\n``` fennel\n(apply conj (hash-map) [:c 3] [[:a 1] [:b 2]])\n;; => {:a 1 :b 2 :c 3}\n```"
    },
    [<function 217>] = {
      ["fnl/arglist"] = { "([x])", "([x y])", "([x y & xs])" },
      ["fnl/docstring"] = "Deep compare values.\n\n# Examples\n\n`eq' can compare both primitive types, tables, and user defined types\nthat have `__eq` metamethod.\n\n``` fennel\n(assert-is (eq 42 42))\n(assert-is (eq [1 2 3] [1 2 3]))\n(assert-is (eq (hash-set :a :b :c) (hash-set :a :b :c)))\n(assert-is (eq (hash-set :a :b :c) (ordered-set :c :b :a)))\n```\n\nDeep comparison is used for tables which use tables as keys:\n\n``` fennel\n(assert-is (eq {[1 2 3] {:a [1 2 3]} {:a 1} {:b 2}}\n               {{:a 1} {:b 2} [1 2 3] {:a [1 2 3]}}))\n(assert-is (eq {{{:a 1} {:b 1}} {{:c 3} {:d 4}} [[1] [2 [3]]] {:a 2}}\n               {[[1] [2 [3]]] {:a 2} {{:a 1} {:b 1}} {{:c 3} {:d 4}}}))\n```"
    },
    [<function 218>] = {
      ["fnl/arglist"] = { "[multimethod]" },
      ["fnl/docstring"] = "Given a `multimethod', returns a map of dispatch values -> dispatch fns"
    },
    [<function 219>] = {
      ["fnl/arglist"] = { "[n", "col]" },
      ["fnl/docstring"] = "Returns a sequence of the first `n' items in `col', or all items if\nthere are fewer than `n'."
    },
    [<function 220>] = {
      ["fnl/arglist"] = { "\n  ([f args])", "\n  ([f a args])", "\n  ([f a b args])", "\n  ([f a b c args])", "\n  ([f a b c d & args])" },
      ["fnl/docstring"] = "Apply `f' to the argument list formed by prepending intervening\narguments to `args', and `f' must support variadic amount of\narguments.\n\n# Examples\nApplying `add' to different amount of arguments:\n\n``` fennel\n(assert-eq (apply add [1 2 3 4]) 10)\n(assert-eq (apply add 1 [2 3 4]) 10)\n(assert-eq (apply add 1 2 3 4 5 6 [7 8 9]) 45)\n```"
    },
    [<function 221>] = {
      ["fnl/arglist"] = { "attr-map?", "name", "expr" }
    },
    [<function 222>] = {
      ["fnl/arglist"] = { "[x]" },
      ["fnl/docstring"] = "Returns its argument."
    },
    [<function 223>] = {
      ["fnl/arglist"] = { "([])", "([x])" },
      ["fnl/docstring"] = "Test if `x' is nil."
    },
    [<function 224>] = {
      ["fnl/arglist"] = { "[f]" },
      ["fnl/docstring"] = "Takes a function `f' and returns the function that takes the same\namount of arguments as `f', has the same effect, and returns the\noppisite truth value."
    },
    [<function 225>] = {
      ["fnl/arglist"] = { "a", "b", "..." },
      ["fnl/docstring"] = "Comparison operator; works the same as Lua but accepts more arguments."
    },
    [<function 226>] = {
      ["fnl/arglist"] = { "[col]" },
      ["fnl/docstring"] = "Create sequential table.\n\nTransforms original table to sequential table of key value pairs\nstored as sequential tables in linear time.  If `col' is an\nassociative table, returns sequential table of vectors with key and\nvalue.  If `col' is sequential table, returns its shallow copy.  If\n`col' is string, return sequential table of its codepoints.\n\n# Examples\nSequential tables remain as is:\n\n``` fennel\n(seq [1 2 3 4])\n;; [1 2 3 4]\n```\n\nAssociative tables are transformed to format like this `[[key1 value1]\n... [keyN valueN]]` and order is non deterministic:\n\n``` fennel\n(seq {:a 1 :b 2 :c 3})\n;; [[:b 2] [:a 1] [:c 3]]\n```\n\nSee `into' macros for transforming this back to associative table.\nAdditionally you can use `conj' and `apply' with\n`hash-map':\n\n``` fennel\n(apply conj (hash-map) [:c 3] [[:a 1] [:b 2]])\n;; => {:a 1 :b 2 :c 3}\n```"
    },
    [<function 227>] = {
      ["fnl/arglist"] = { "([])", "([f])", "([f g])", "([f g & fs])" },
      ["fnl/docstring"] = "Compose functions."
    },
    [<function 228>] = {
      ["fnl/arglist"] = { "[binding test]", "&", "body" }
    },
    [<function 229>] = {
      ["fnl/arglist"] = { "binding1", "module-name1", "..." },
      ["fnl/docstring"] = "Binds a table of macros from each macro module according to a binding form.\n  Each binding form can be either a symbol or a k/v destructuring table.\n  Example:\n    (import-macros mymacros                 :my-macros    ; bind to symbol\n                   {:macro1 alias : macro2} :proj.macros) ; import by name"
    },
    [<function 230>] = {
      ["fnl/arglist"] = { "[col", "n]" },
      ["fnl/docstring"] = "Returns the nth rest of `col', `col' when `n' is 0.\n\n# Examples\n\n``` fennel\n(assert-eq (nthrest [1 2 3 4] 3) [4])\n(assert-eq (nthrest [1 2 3 4] 2) [3 4])\n(assert-eq (nthrest [1 2 3 4] 1) [2 3 4])\n(assert-eq (nthrest [1 2 3 4] 0) [1 2 3 4])\n```\n"
    },
    [<function 231>] = {
      ["fnl/arglist"] = { "[x]" },
      ["fnl/docstring"] = "Test if `x' is `false'"
    },
    [<function 232>] = {
      ["fnl/arglist"] = { "x1", "x2", "..." },
      ["fnl/docstring"] = "Bitwise OR of any number of arguments.\nOnly works in Lua 5.3+ or LuaJIT with the --use-bit-lib flag."
    },
    [<function 233>] = {
      ["fnl/arglist"] = { "[x]" },
      ["fnl/docstring"] = "Check if collection is empty."
    },
    [<function 234>] = {
      ["fnl/arglist"] = { "[col]" },
      ["fnl/docstring"] = "Transforms any table `col' to key-value sequence."
    },
    [<function 235>] = {
      ["fnl/arglist"] = { "[pred", "tbl]" },
      ["fnl/docstring"] = "Test if any item in `tbl' satisfies the `pred'."
    },
    [<function 236>] = {
      ["fnl/arglist"] = { "[n", "col]" },
      ["fnl/docstring"] = "Returns a sequence of the first `n' items in `col', or all items if\nthere are fewer than `n'."
    },
    [<function 237>] = {
      ["fnl/arglist"] = { "[key value (iterator)]", "..." },
      ["fnl/body-form?"] = true,
      ["fnl/docstring"] = "Runs the body once for each set of values provided by the given iterator.\nMost commonly used with ipairs for sequential tables or pairs for  undefined\norder, but can be used with any iterator."
    },
    [<function 238>] = {
      ["fnl/arglist"] = { "[tbl]" },
      ["fnl/docstring"] = "Returns table with same items as in `tbl' but in reverse order."
    },
    [<function 239>] = {
      ["fnl/arglist"] = { "([n col])", "([n step col])", "([n step pad col])" },
      ["fnl/docstring"] = "Returns a sequence of sequences of `n' items each, at offsets step\napart. If `step' is not supplied, defaults to `n', i.e. the partitions\ndo not overlap. If a `pad' collection is supplied, use its elements as\nnecessary to complete last partition up to `n' items. In case there\nare not enough padding elements, return a partition with less than `n'\nitems.\n\n# Examples\nPartition sequence into sub-sequences of size 3:\n\n``` fennel\n(assert-eq (partition 3 [1 2 3 4 5 6]) [[1 2 3] [4 5 6]])\n```\n\nWhen collection doesn't have enough elements, partition will not include those:\n\n``` fennel\n(assert-eq (partition 3 [1 2 3 4]) [[1 2 3]])\n```\n\nPartitions can overlap if step is supplied:\n\n``` fennel\n(assert-eq (partition 2 1 [1 2 3 4]) [[1 2] [2 3] [3 4]])\n```\n\nAdditional padding can be used to supply insufficient elements:\n\n``` fennel\n(assert-eq (partition 3 3 [3 2 1] [1 2 3 4]) [[1 2 3] [4 3 2]])\n```"
    },
    [<function 240>] = {
      ["fnl/arglist"] = { "val", "..." },
      ["fnl/docstring"] = "Thread-last macro.\n  Same as ->, except splices the value into the last position of each form\n  rather than the first."
    },
    [<function 241>] = {
      ["fnl/arglist"] = { "[f", "val", "tbl]" },
      ["fnl/docstring"] = "Reduces an associative table using function `f' and initial value `val'.\n\n`f' should be a function of 3 arguments.  Returns the result of\napplying `f' to `val', the first key and the first value in `tbl',\nthen applying `f' to that result and the 2nd key and value, etc.  If\n`tbl' contains no entries, returns `val' and `f' is not called.  Note\nthat reduce-kv is supported on sequential tables and strings, where\nthe keys will be the ordinals.\n\nEarly termination is possible with the use of `reduced'\nfunction.\n\n# Examples\nReduce associative table by adding values from all keys:\n\n``` fennel\n(local t {:a1 1\n          :b1 2\n          :a2 2\n          :b2 3})\n\n(reduce-kv #(+ $1 $3) 0 t)\n;; => 8\n```\n\nReduce table by adding values from keys that start with letter `a':\n\n``` fennel\n(local t {:a1 1\n          :b1 2\n          :a2 2\n          :b2 3})\n\n(reduce-kv (fn [res k v] (if (= (string.sub k 1 1) :a) (+ res v) res))\n           0 t)\n;; => 3\n```"
    },
    [<function 242>] = {
      ["fnl/arglist"] = { "([f col])", "([f val col])" },
      ["fnl/docstring"] = "Reduce collection `col' using function `f' and optional initial value `val'.\n\n`f' should be a function of 2 arguments.  If val is not supplied,\nreturns the result of applying f to the first 2 items in coll, then\napplying f to that result and the 3rd item, etc.  If coll contains no\nitems, f must accept no arguments as well, and reduce returns the\nresult of calling f with no arguments.  If coll has only 1 item, it is\nreturned and f is not called.  If val is supplied, returns the result\nof applying f to val and the first item in coll, then applying f to\nthat result and the 2nd item, etc.  If coll contains no items, returns\nval and f is not called.  Calls `seq' on `col'.\n\nEarly termination is possible with the use of `reduced'\nfunction.\n\n# Examples\nReduce sequence of numbers with `add'\n\n``` fennel\n(reduce add [1 2 3 4])\n;; => 10\n(reduce add 10 [1 2 3 4])\n;; => 20\n```"
    },
    [<function 243>] = {
      ["fnl/arglist"] = { "[index start stop step?]", "..." },
      ["fnl/body-form?"] = true,
      ["fnl/docstring"] = "Numeric loop construct.\nEvaluates body once for each value between start and stop (inclusive)."
    },
    <metatable> = {
      __index = {
        get = <function 244>,
        set = <function 245>,
        setall = <function 246>
      },
      __mode = "k"
    }
  }
}

[rktjmp]

Hm, yeah I tried pretty hard to recreate it. When testing in docker you have to run nvim twice because the first time Fennel is loaded to compile hotpot itself and your config, the second time Fennel isn't loaded until you specifically require it.

Note: the examples here are done with :lua so :Fnl doesn't import Fennel before testing! All run in a fresh docker container.

---

In this video:

• load nvim
• loads from cache (fennel isn't required)
• print package.preload, includes "fennel" via provide_require_fennel helper
• require fennel
• see searcher look for fennel, hit the provide preloader, require our vendored fennel file and then search for a bunch of fennel modules (this is fennel.lua's doing).
• package.preload now includes a bunch of fennel stuff, all working as it should.

edit: streamable link if GH wont serve video https://streamable.com/a2ib57

fennel-req-2021-09-26_15.55.31.mp4

---

In this video:

• load nvim
• loads from cache
• (all in one line, might want to pause the video) require fennel (via provides_...), does the normal loading stuff as in the previous video, then access .view(...) and print the object.
• All working as it should?

https://streamable.com/to899l

fennel-req2-2021-09-26_15.57.36.mp4

So I think it must be something in your config that's fiddling with something? Maybe cljlib is doing something unexpected?

[rktjmp]

In fact it's even stranger that it only doesn't work in macros for you since being in a macro implies that Fennel must have been loaded already - to compile the macro.

It may be another sandbox issue perhaps, but my tests in Docker (in and out of a macro file) worked fine with just the default Hotpot options save for provide_require_fennel = true.

[datwaft]

Maybe related to the patches for using .so files or the like.

I will do more testing tomorrow, thanks for your help!