Add ClojureScript Support

[furkan3ayraktar]

We have discussed several times how to best support ClojureScript. You can see the earlier discussion here and the follow-up discussion here. Ultimately, we decided to implement the simplest possible solution to give Polylith users the best experience and flexibility while working with ClojureScript or mixed Clojure/ClojureScript workspaces. We want to support all different kinds of use cases, but at the same time, we want to come up with a simple solution to make it easier for us to maintain and explain it quickly to newcomers. Regardless of your current choice of tooling and setup, you should be able to adopt Polylith in your projects using Clojure and/or ClojureScript.

After talking to many community members, we decided to keep Polylith simple and not introduce multiple workspace support. You can see that discussion and the reasoning behind it here. This led us to find a solution in which we consider a brick a brick regardless of whether it has Clojure, ClojureScript, or both, as sources. In our solution, clj, cljs, and cljc files can live side by side, in any combination, inside a base or component. It’s your choice whether you want to mix dialects within a brick or to have pure Clojure and ClojureScript bricks. We think it’s essential to support all these use cases.

Another key decision we made, after much thorough thinking, was not to provide a default test runner for ClojureScript. In many cases, ClojureScript developers have a file system watcher running in the background, continuously running the tests in the browser and the NodeJS environment. Great tools exist around this, shadow-cljs being one of the examples worth mentioning. It will still be possible to support ClojureScript testing via external test runners.

We, as a team, try to avoid introducing breaking changes. To keep it that way, you will need to tell the poly tool to support ClojureScript explicitly. One thing that we couldn’t avoid was related to external test runners. If the ClojureScript support is enabled for a workspace, the external test runners will break because they will try to run ClojureScript tests unless they explicitly exclude them. We updated the default Clojure test runner so that it ignores ClojureScript tests. It is an easy fix, and external test runner maintainers should be able to address it quickly. Also, please note that if you do not enable ClojureScript support for the workspace, everything will work as before (including external test runners). This is how we update the default Clojure test runner to exclude cljs test namespaces:

polylith/components/clojure-test-test-runner/src/polylith/clj/core/clojure_test_test_runner/core.clj

Lines 13 to 32 in 6ca059f

	(defn- clj-namespace? [{:keys [file-path]}]
	(or (str/ends-with? file-path ".clj")
	(str/ends-with? file-path ".cljc")))
	(defn brick-test-namespaces [bricks test-brick-names]
	(let [brick-name->namespaces (->> bricks
	(map (juxt :name #(-> % :namespaces :test)))
	(into {}))]
	(->> test-brick-names
	(mapcat brick-name->namespaces)
	(filter clj-namespace?)
	(map :namespace)
	(into []))))
	(defn project-test-namespaces [project-name projects-to-test namespaces]
	(when (contains? (set projects-to-test) project-name)
	(->> (:test namespaces)
	(filter clj-namespace?)
	(map :namespace)
	(into []))))

To enable ClojureScript support, you will add a new option called :dialects in your workspace.edn. We expect a set or a vector of dialects as string values there. Currently, "clj" and "cljs" are the only allowed values. .cljc files are always included regardless of the dialect. If :dialects is left out, it will be set to #{"clj"}. You can see an example here. You can clone the cljs-frontend branch of the Realworld example project and run clojure -M:poly to test the ClojureScript support.

We still have a few more things that we’d like to implement. Those are:

• We want to read package.json files to list NPM dependencies in the libs command.
• We plan to create an example development environment for a mixed Clojure/ClojureScript project using shadow-cljs. This will allow the developer to run a single command to get one Clojure and one ClojureScript REPL, including all their code from the workspace.
• We plan to update the create command to choose whether to create clj or cljs files. This may be implemented using the Selmer templating support.

When ClojureScript support transitions from experimental to final state, we will update all the necessary parts of the documentation.

It would be helpful if the community tested this version. You can see the current implementation in this PR. You can already start using it by:

 :aliases {:poly {:extra-deps {polylith/clj-poly
                               {:git/url   "https://github.com/polyfy/polylith.git"
                                :sha       "7fa6964e08cdf0f505af8b1cdcb99bc65a55d389"
                                :deps/root "projects/poly"}}
                  :main-opts  ["-m" "polylith.clj.core.poly-cli.core"]}}

As always, we are looking forward to your feedback.

Joakim & Furkan

[seancorfield]

I've updated my external test runner to leverage this in a PR and I'll track dialect progress.

[farcaller]

Trying it out and it's pretty good! Here's one questionable issue that I noticed so far:

In a pb_mucklet component I have the following ifc.cljc:

(ns bankng.pb-mucklet.ifc
  #?(:clj (:require [bankng.pb-mucklet.client :as client])
     :cljs (:require [bankng.pb-mucklet.server :as server])))

With the following client:

(ns bankng.pb-mucklet.client
  (:require [bankng.proto-tools.ifc :refer [map->proto proto->map]]
            [mount.core :refer [defstate]]
            [bankng.config.ifc :refer [config]])
  (:import [io.grpc Grpc InsecureChannelCredentials ManagedChannel]
           [bankng.mucklet LookupCharacterRequest MuckletGrpc SendMessageRequest MuckletGrpc$MuckletBlockingStub]))

poly info will report Error 107: Missing components in the mucklet-proxy project for these interfaces: config, proto-tools. Now, I know mucklet-proxy project is clojurescript only, but poly doesn't. I suppose, adding the missing components isn't a huge deal, but it'd be nicer if a project could be declared as clj/cljs only so that the extra deps wouldn't need to be resolved.

[furkan3ayraktar]

Hmm, I understand your point. We decided to keep it simple. If a component depends on other components, regardless of their dialect, they will be dependencies. This greatly reduces the complexity on our side and removes the necessity to declare bricks/projects as clj/cljs on the user side.

I know it is cool to be able to import clj/cljs with reader conditionals, but it can be cleaner to separate them into different components and require them from clj/cljs projects separately. Of course, alternatively, you can keep it as it is and add the components, unnecessarily, to the project to avoid the warning.

Do you think this is an okay workaround or would you say we should reconsider our decision on this?

[farcaller]

I don't think splitting the component in my case is a reasonable workaround, here's why. pb-mucklet is a component that has a protobuf definition in its resources. It also has the generated java code in components/pb-mucklet/java-src and the generated js in components/pb-mucklet/src/gen. Its deps.edn also has target/classes in the paths.

The component itself is a thin wrapped around the protobuf code. It contains the java side by having the classes in a component-local dir and adding it to the classpath. It also contains the js code by having it inside src and requireing it from there.

It doesn't seem to be possible to require js files across different components (at least with shadow-cljs), so the only way i see it could be restructured is to have 3 components: 1 for the resource and two for the generated code, which sounds like an overkill to me.

Do you think I'm structuring it in a suboptimal way or this case is legit?

[furkan3ayraktar]

I would split them into more components rather than coupling generated JS and Java sources together since they are never used at the same time. It would make it more understandable, imo, and avoid the problem you are having in the first place.

[seancorfield]

How will dialects affect the create command? Right now, those always generate .clj files (plus deps.edn etc).

The existing Selmer-based template code (merged to master) still has a hardwired set of files that get generated for each create type, and that uses a hardwired mapping from templates to target generated files, e.g., test.clj => components/<brick>/test/<top-ns>/<brick>_test.clj

Is the current thinking for this "simplest possible solution" to leave that as-is (and require users change the file extensions as needed) or that a create command will now accept a dialect and use that to affect the target generated file extension (for source code)? Would the source templates still be .clj or would those also need to match the dialect?

[seancorfield]

Based on some discussions elsewhere, it sounds like create will accept dialect:cljs as an option at some point, and have templates for the (minimal) set of files it would create per-dialect -- for a closed set of dialects (#{"clj" "cljs"}).

[tengstrand]

Now we also generates package.json files if cljs is choosen, see Templates.

[tengstrand]

The create command is now supporting e.g. dialect:cljs which is also passed to the Selmer template engine in the cljs-support branch. @seancorfield