Require and providing source files

Require and load

To effectively use require with replumb and all other cljs.js-dependent code, something more needs to be said.

First of all, when a require is triggered, replumb, actually following its predecessors ambly and planck, converts it to a ns call against the current namespace, so that cljs.user=> (require 'foo.bar.baz) evaluates (ns cljs.user (:require 'foo.bar.baz) instead.

There is a valuable reason to do this: ns is very solid and already handles things like loading dependent namespaces and option parsing. What happens next is not very surprising: the namespace needs to be mapped to a file, this file needs to be loaded somehow and forms need to be evaluated (for example def and defn will define vars).

Duly, cljs.js abstracts IO through what is called the *load-fn*. This is better explained in its docstring, whose copy you will find basically everywhere in replumb.core (always double check for an updated version upstream):

Each runtime environment provides a different way to load a library. Whatever function *load-fn* is bound to will be passed two arguments - a map and a callback function: The map will have the following keys:

:name   - the name of the library (a symbol)
:macros - modifier signalling a macros namespace load
:path   - munged relative library path (a string)

It is up to the implementer to correctly resolve the corresponding .cljs, .cljc, or .js resource (the order must be respected). If :macros is true resolution should only consider .clj or .cljc resources (the order must be respected). Upon resolution the callback should be invoked with a map containing the following keys:

 :lang       - the language, :clj or :js
 :source     - the source of the library (a string)
 :cache      - optional, if a :clj namespace has been precompiled to :js, can
               give an analysis cache for faster loads.
 :source-map - optional, if a :clj namespace has been precompiled to :js, can
           give a V3 source map JSON

If the resource could not be resolved, the callback should be invoked with nil.

Replumb takes the same approach, but strives to do better. We are building a layer on top of cljs.js after all and therefore we have to provide more sugar.

First of all, since version 0.1.3 a new option to replumb.core/read-eval-call has been added. This option entirely replaces cljs.js/*load-fn* doing the rebinding for you. The name is :load-fn! and whenever is present it has precedence over any other *load-fn*. Use with care.

More interestingly, replumb can be customized with what is the very basis of loading: a function that given a file path, returns its content. This eases the burden of adhere with the protocol above.

The name of the option is :read-file-fn!, an asynchronous 2-arity function with signature [file-path src-cb]: this function will always receive the file path complete with the right extension so that client code can forget about order and *load-fn* callback. There is another callback here, src-cb, but it is easier to understand. It should just be called with either the file content or nil as argument.

An example of implementation can be found in either the replumb.browser.io or the replumb.nodejs.io namespace. Moreover, the two handy replumb.core/browser-options and replumb.core/nodejs-options were also born to provide an hopefully straightforward external API for it.

Providing source files

The option :read-file-fn! is not sufficient alone to provide a working *load-fn*. We need a second brand new option, :src-paths, which accepts a sequence of strings representing file paths. It has to be sequential and contain strings or no *load-fn* will be added, resulting in the dreaded "No *load-fn* set" error.

This opens up another big question, that is how to provide the source files to our environment. It should be now clear that in order for this to work:

cljs.user=> (require 'clojure.string)
nil
cljs.user=> (doc clojure.string/trim)
-------------------------
clojure.string/trim
([s])
  Removes whitespace from both ends of string.
nil

The file clojure/string.cljs should be available in :src-paths or replumb won't be able to employ its :read-file-fn! on it in order to read the docstring for trim.

One of the biggest trade-offs of embedding a REPL in your client-side JavaScript app is size. You need a plethora of source files for doc, source and other REPL perks to work. For instance, in the particular case of clojure.string you need the whole Google Closure library. The reason is obvious if we peek under the carpet: goog.string is imported as dependency, and might potentially import other Google Closure libraries. Another recent issue showed that even clojure/core.clj is needed by some symbol.

One solution could be to include the out folder, result of the compilation, in :src-paths. This works, and it is exactly what browser-repl and node-repl are doing at the moment. The problem is that these files were just our app dependencies. They are still missing, for instance, the Clojure core namespaces that were not explicitly required. So a better solution still needs to be thought of.

An idea is to centralize on a (possibly distributed) remote location the source files that all the ClojureScript REPL-ish app need. A little bit like Clojars, which by the way is going through its crowd-funding campaign, this can another a successful Clojure(Script) story to tell.