CODEWITHSUNDEEP
clojuredeps-edn
Erp12
Erp12

Reputation: 652

How to compile all source and tests before running -M or -X from deps.edn?

My project has some code that requires AOT compilation.

Below is a shortened version of my deps.edn. The only way I have found to compile some code before each test run is to add a -e (compile,'my.project.namespace) pair to the :main-opts. This requires knowledge of the exact namespaces that require compilation, which changes periodically during development. Even simple experimentation in a unit test may require a temporary change to the deps.edn which is very annoying.

{:paths   ["src" "classes"]
 :deps    {}
 :aliases {:test           {:extra-paths ["test"]}
           :test/cognitect {:extra-deps {com.cognitect/test-runner {:git/url "..." :sha "..."}}
                            :main-opts  ["-e" "(compile,'my.project.namespace1)"
                                         "-e" "(compile,'my.project.namespace2)"
                                         "-e" "(compile,'my.project.namespace3)"
                                         "-m" "cognitect.test-runner"]
                            :exec-fn    cognitect.test-runner.api/test}}

This would be invoked with clj -M:test:test/cognitect.

Is there an easy way to compile the entire project (including tests) when -M (or -X) is invoked with a certain alias?

I know that -e can handle any Clojure expression so one option would be to write an entire mini program that will enumerate all the namespaces and call compile on them. I would be okay with this as long as the code is simple. Is this a good idea?

In general, how would one setup a deps.edn to AOT compile the entire project?

Upvotes: 1

Views: 803

Answers (2)

Erp12
Erp12

Reputation: 652

Sean Corfield's answer seems to be the best direction. Given how new tools.build is I thought it might be helpful to expand a bit for others to see how this would be done and highlight an issue that I encountered.

(def class-dir "target/classes")

(def basis (b/create-basis {:project "deps.edn"
                            ;; Include any aliases that will bring deps required for 
                            ;; compilation onto the classpath.
                            :aliases [:spark-deps :test]}))

(defn compile-for-tests
  []
  (b/compile-clj {:basis basis
                  :src-dirs ["src" "test"]
                  :class-dir class-dir
                  ;; Filters which classes get written into `class-dir` by their namespace prefix.
                  :filter-nses ['my.project]})`)

I beleive that the above is typically all that you need, however my project requires 2 phases of compilation. One to compile some :gen-class namespaces and a second to compile the rest of the codebase (some of which imports those :gen-class classes as a Java class).

Also, those :gen-class namespaces seem to need to be compiled one at a time (even though they don't reference each other) in order to not get ClassNotFoundException. I can't explain why this is, but I will update this answer if I ever figure it out. Below is the exact function I am currently using.

(defn ns-under-prefix
  [prefix paths]
  (let [all-ns (mapcat #(-> % b/resolve-path jio/file find/find-namespaces-in-dir) paths)]
    (filter #(str/starts-with? (name %) (name prefix)) all-ns)))


(defn compile-for-tests
  [_]
  ;; Compiling one at a time is required... for some reason.
  (doseq [nmspc (ns-under-prefix 'erp12.clark.core.aot ["src"])]
    (println "Compiling" nmspc)
    (b/compile-clj {:basis basis
                    :src-dirs ["src"]
                    :class-dir class-dir
                    :ns-compile [nmspc]}))
  (println "Compiled AOT namespaces.")
  ;; Compile everything else. Relies on the AOT classes already existing.
  (b/compile-clj {:basis basis
                  :src-dirs ["src" "test"]
                  :class-dir class-dir
                  :filter-nses ['erp12.clark.core]}))

Upvotes: 1

Sean Corfield
Sean Corfield

Reputation: 6666

Take a look at tools.build: https://clojure.org/guides/tools_build

This is the core team's answer to this problem.

Upvotes: 2

Related Questions