Confusing set-env! semantics for different keys about boot HOT 2 CLOSED

Hi! Thanks for opening this ticket!

I agree that set-env! is not ideal in this case. Resource and source paths should be treated the same, so that is indeed a bug which needs to be fixed. However, there is a deeper problem here: source and resource paths are on the class path. This means that there is no way to remove them or replace them with anything else, because there's no mechanism we can use to remove a directory from the classloader's class path (as far as I know). This is why set-env! always appends to :src-paths.

The set-env! function modifies the boot environment, which is basically the classloader bootstrapping environment, i.e. classpath configuration, Aether/Pomegranate configuration, JAR dependencies, etc. This env data cannot contain anything that can't be round-tripped through the Clojure reader and printer because the env must be able to be communicated across pod boundaries (pods are isolated, separate Clojure runtimes that can run concurrently in boot to perform work). Since functions are not serializable they can't be valid values for keys in the env. We exploit this in set-env!: if the value is a function then we apply the function to the current value to obtain the next value.

For example:

(let [deps (get-env :dependencies)]
  (set-env! :dependencies (conj deps '[foo/bar "1.2.3"])))

is equivalent to

(set-env! :dependencies #(conj % '[foo/bar "1.2.3"]))

I've been working on the boot-clj/boot tempdirs branch which addresses the classpath problems discussed above. (There are other complications from this classpath issue in other parts of boot also, so it's something we want to get right.) In this branch the source and resource paths are not on the classpath. Instead the contents of these directories are copied into boot's classpath as necessary by a background thread. This means that you can add or remove paths from the set of paths this background thread is syncing with the boot classpath, effectively adding or removing directories from the boot classpath at will. This, combined with some other refactoring greatly simplifies the set-env! semantics, removing the need for merge-env! in most cases. This branch also cleans up the names of things a lot (like :source-paths, :resource-paths, etc.) and adds support for the new immutable fileset machinery we've been working on.

The idea behind merge-env! and on-env! are to manage side-effects related to manipulations of the JVM environment. For instance, add-dependencies is called in merge-env! because it might raise an exception. If adding dependencies fails we don't want to add those deps to the :dependencies key of the env atom–we want to just propagate the exception (like if you're in the REPL and you add a bogus dependency, you should be able to retry without needing to clean up your env). On the other hand, sometimes we need to do things after the env has been modified successfully, which is where we use on-env!.

I'll be working on full documentation for the tempdirs stuff over the next few days, in the wiki. I hope to get the docs complete before pushing any breaking changes to master. Your feedback is greatly appreciated!

from boot.

I see, thank you.

from boot.