luchiniatwork / cambada Goto Github PK

View Code? Open in Web Editor NEW

221.0 221.0 28.0 48 KB

Packager for Clojure based on deps.edn (AKA tools.deps). Supporting jar, uberjar and GraalVM's native-image.

License: MIT License

Clojure 100.00%

clojure graalvm packager tools tools-deps

cambada's People

Contributors

Stargazers

Watchers

cambada's Issues

Properly attribute code to leiningen

I noticed the layout of the namespaces, the algorithms, and even the symbol naming in this repository bears uncanny resemblance to leiningen's code.

I would suggest that you ensure that you have @technomancy's permission, that you attribute the maintainers in the Copyright clause, and that you ensure you can redistribute EPL derivative works under an MIT license (this might be alright, but I really have no idea).

I really like the approach you've taken with this library, and I look forward to using it as soon as I'm sure that these issues have been addressed.

Examples:

lein ns: leiningen.jar

(defmethod copy-to-jar :path [project jar-os acc spec]
  (let [root-file (io/file (:path spec))
        root-dir-path (unix-path (dir-string root-file))
        paths (for [child (file-seq root-file)
                    :let [path (relativize-path
                                 (full-path child (unix-path (str child)))
                                 root-dir-path)]]
                (when-not (or (skip-file? child path root-file
                                          (:jar-exclusions project)
                                          (:jar-inclusions project))
                              (added-file? child path acc))
                  (put-jar-entry! jar-os child path)
                  path))]
    (into acc paths)))

(defmethod copy-to-jar :paths [project jar-os acc spec]
  (reduce (partial copy-to-jar project jar-os) acc
          (for [path (:paths spec)]
            {:type :path :path path})))

(defmethod copy-to-jar :bytes [project jar-os acc spec]
  (let [path (unix-path (:path spec))]
    (when-not (some #(re-find % path) (:jar-exclusions project))
      (.putNextEntry jar-os (JarEntry. path))
      (let [bytes (if (string? (:bytes spec))
                    (.getBytes (:bytes spec))
                    (:bytes spec))]
        (io/copy (ByteArrayInputStream. bytes) jar-os)))
    (conj acc path)))

(defmethod copy-to-jar :fn [project jar-os acc spec]
  (let [f (eval (:fn spec))
        dynamic-spec (f project)]
    (copy-to-jar project jar-os acc dynamic-spec)))

cambada ns: cambada.jar

(defmethod copy-to-jar :path [project jar-os acc spec]
  (let [root-file (io/file (:path spec))
        root-dir-path (utils/unix-path (utils/dir-string root-file))
        paths (for [child (file-seq root-file)
                    :let [path (utils/relativize-path
                                (utils/full-path child (utils/unix-path (str child)))
                                root-dir-path)]]
                (when-not (or (skip-file? child path root-file
                                          (:jar-exclusions project)
                                          (:jar-inclusions project))
                              (added-file? child path acc))
                  (put-jar-entry! jar-os child path)
                  path))]
    (into acc paths)))

(defmethod copy-to-jar :paths [project jar-os acc spec]
  (reduce (partial copy-to-jar project jar-os) acc
          (for [path (:paths spec)]
            {:type :path :path path})))

(defmethod copy-to-jar :bytes [project jar-os acc spec]
  (let [path (utils/unix-path (:path spec))]
    (when-not (some #(re-find % path) (:jar-exclusions project))
      (.putNextEntry jar-os (JarEntry. path))
      (let [bytes (if (string? (:bytes spec))
                    (.getBytes (:bytes spec))
                    (:bytes spec))]
        (io/copy (ByteArrayInputStream. bytes) jar-os)))
    (conj acc path)))

(defmethod copy-to-jar :fn [project jar-os acc spec]
  (let [f (eval (:fn spec))
        dynamic-spec (f project)]
    (copy-to-jar project jar-os acc dynamic-spec)))

lein ns: leiningen.uberjar

(defn- merger-match? [[pattern] filename]
  (boolean
   (condp instance? pattern
     String (= pattern filename)
     Pattern (re-find pattern filename))))

(def ^:private skip-merger
  [(constantly ::skip)
   (constantly nil)])

(def ^:private default-merger
  [(fn [in out file prev]
     (when-not prev
       (.setCompressedSize file -1)
       (.putNextEntry out file)
       (io/copy (.getInputStream in file) out)
       (.closeEntry out))
     ::skip)
   (constantly nil)])

(defn- make-merger [fns]
  {:pre [(sequential? fns) (= 3 (count fns)) (every? ifn? fns)]}
  (let [[read-fn merge-fn write-fn] fns]
    [(fn [in out file prev]
       (with-open [ins (.getInputStream in file)]
         (let [new (read-fn ins)]
           (if-not prev
             new
             (merge-fn new prev)))))
     (fn [out filename result]
       (.putNextEntry out (ZipEntry. filename))
       (write-fn (CloseShieldOutputStream. out) result)
       (.closeEntry out))]))

cambada ns: cambada.uberjar

(defn ^:private  merger-match? [[pattern] filename]
  (boolean
   (condp instance? pattern
     String (= pattern filename)
     Pattern (re-find pattern filename))))

(def ^:private default-merger
  [(fn [in out file prev]
     (when-not prev
       (.setCompressedSize file -1)
       (.putNextEntry out file)
       (io/copy (.getInputStream in file) out)
       (.closeEntry out))
     ::skip)
   (constantly nil)])

(defn ^:private make-merger [fns]
  {:pre [(sequential? fns) (= 3 (count fns)) (every? ifn? fns)]}
  (let [[read-fn merge-fn write-fn] fns]
    [(fn [in out file prev]
       (with-open [ins (.getInputStream in file)]
         (let [new (read-fn ins)]
           (if-not prev
             new
             (merge-fn new prev)))))
     (fn [out filename result]
       (.putNextEntry out (ZipEntry. filename))
       (write-fn (CloseShieldOutputStream. out) result)
       (.closeEntry out))]))

(defn ^:private map-vals
  "Like 'update', but for all values in a map."
  [m f & args]
  (zipmap (keys m) (map #(apply f % args) (vals m))))

(def ^:private skip-merger
  [(constantly ::skip)
   (constantly nil)])

lein ns: leiningen.clean

(defn real-directory?
  "Returns true if this file is a real directory, false if it is a symlink or a
  normal file."
  [f]
  (if (= :windows (utils/get-os))
    (.isDirectory f)
    (and (.isDirectory f)
         (not (utils/symlink? f)))))

(defn delete-file-recursively
  "Delete file f. If it's a directory, recursively delete all its contents.
  Raise an exception if any deletion fails unless silently is true."
  [f & [silently]]
  (let [f (io/file f)]
    (when (real-directory? f)
      (doseq [child (.listFiles f)]
        (delete-file-recursively child silently)))
    (.setWritable f true)
    (io/delete-file f silently)))

cambada ns: cambada.utils (cambada.clean is basically a proxy to these functions)

(defn real-directory?
  "Returns true if this file is a real directory, false if it is a symlink or a
  normal file."
  [f]
  (if (= :windows (get-os))
    (.isDirectory f)
    (and (.isDirectory f)
         (not (symlink? f)))))

(defn delete-file-recursively
  "Delete file f. If it's a directory, recursively delete all its contents.
  Raise an exception if any deletion fails unless silently is true."
  [f & [silently]]
  (let [f (io/file f)]
    (when (real-directory? f)
      (doseq [child (.listFiles f)]
        (delete-file-recursively child silently)))
    (.setWritable f true)
    (io/delete-file f silently)))

Classes from clojure.data.xml can not be found in produced uberjar

Dependencies present in cambada and compiled project never appear in the list of compiled classes due to the fact that they are already loaded into the classloader before the compile function is called.

To fix that separate classloader should be created.

uberjar missing deps

Using deps.edn the uberjar ends up missing deps.

when my deps looks like:

:deps {org.clojure/clojure {:mvn/version "1.11.1"}
        org.clojure/tools.cli {:mvn/version "1.0.206"}
        com.github.mwiede/jsch {:mvn/version "0.2.3"}
        nrepl/bencode {:mvn/version "1.1.0"}}

$ clj -X:deps tree
org.clojure/clojure 1.11.1
  . org.clojure/spec.alpha 0.3.218
  . org.clojure/core.specs.alpha 0.2.62
org.clojure/tools.cli 1.0.206
com.github.mwiede/jsch 0.2.3
nrepl/bencode 1.1.0

but my uberjar is built without org.clojure/spec.alpha

Creating target/bbssh-0.1.0-SNAPSHOT-standalone.jar
  Including bbssh-0.1.0-SNAPSHOT.jar
  Including jsch-0.2.3.jar
  Including bencode-1.1.0.jar
  Including clojure-1.11.1.jar
  Including tools.cli-1.0.206.jar
  Including core.specs.alpha-0.2.62.jar
Done!

Notice it's missing here. Then when I run the uberjar:

java -cp target/bbssh-0.1.0-SNAPSHOT-standalone.jar clojure.main -m bbssh.core
Exception in thread "main" java.lang.ExceptionInInitializerError
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:398)
	at clojure.lang.RT.classForName(RT.java:2209)
	at clojure.lang.RT.classForName(RT.java:2218)
	at clojure.lang.RT.loadClassForName(RT.java:2237)
	at clojure.lang.RT.load(RT.java:449)
	at clojure.lang.RT.load(RT.java:424)
	at clojure.core$load$fn__6908.invoke(core.clj:6161)
	at clojure.core$load.invokeStatic(core.clj:6160)
	at clojure.core$load.doInvoke(core.clj:6144)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.core$load_one.invokeStatic(core.clj:5933)
	at clojure.core$load_one.invoke(core.clj:5928)
	at clojure.core$load_lib$fn__6850.invoke(core.clj:5975)
	at clojure.core$load_lib.invokeStatic(core.clj:5974)
	at clojure.core$load_lib.doInvoke(core.clj:5953)
	at clojure.lang.RestFn.applyTo(RestFn.java:142)
	at clojure.core$apply.invokeStatic(core.clj:669)
	at clojure.core$load_libs.invokeStatic(core.clj:6016)
	at clojure.core$load_libs.doInvoke(core.clj:6000)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:669)
	at clojure.core$require.invokeStatic(core.clj:6038)
	at clojure.core.server$loading__6789__auto____8961.invoke(server.clj:9)
	at clojure.core.server__init.load(Unknown Source)
	at clojure.core.server__init.<clinit>(Unknown Source)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:398)
	at clojure.lang.RT.classForName(RT.java:2209)
	at clojure.lang.RT.classForName(RT.java:2218)
	at clojure.lang.RT.loadClassForName(RT.java:2237)
	at clojure.lang.RT.load(RT.java:449)
	at clojure.lang.RT.load(RT.java:424)
	at clojure.core$load$fn__6908.invoke(core.clj:6161)
	at clojure.core$load.invokeStatic(core.clj:6160)
	at clojure.core$load.doInvoke(core.clj:6144)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.core$load_one.invokeStatic(core.clj:5933)
	at clojure.core$load_one.invoke(core.clj:5928)
	at clojure.core$load_lib$fn__6850.invoke(core.clj:5975)
	at clojure.core$load_lib.invokeStatic(core.clj:5974)
	at clojure.core$load_lib.doInvoke(core.clj:5953)
	at clojure.lang.RestFn.applyTo(RestFn.java:142)
	at clojure.core$apply.invokeStatic(core.clj:669)
	at clojure.core$load_libs.invokeStatic(core.clj:6016)
	at clojure.core$load_libs.doInvoke(core.clj:6000)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:669)
	at clojure.core$require.invokeStatic(core.clj:6038)
	at clojure.core$require.doInvoke(core.clj:6038)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.lang.Var.invoke(Var.java:384)
	at clojure.lang.RT.doInit(RT.java:491)
	at clojure.lang.RT.init(RT.java:467)
	at clojure.main.main(main.java:38)
Caused by: java.io.FileNotFoundException: Could not locate clojure/spec/alpha__init.class, clojure/spec/alpha.clj or clojure/spec/alpha.cljc on classpath.
	at clojure.lang.RT.load(RT.java:462)
	at clojure.lang.RT.load(RT.java:424)
	at clojure.core$load$fn__6908.invoke(core.clj:6161)
	at clojure.core$load.invokeStatic(core.clj:6160)
	at clojure.core$load.doInvoke(core.clj:6144)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.core$load_one.invokeStatic(core.clj:5933)
	at clojure.core$load_one.invoke(core.clj:5928)
	at clojure.core$load_lib$fn__6850.invoke(core.clj:5975)
	at clojure.core$load_lib.invokeStatic(core.clj:5974)
	at clojure.core$load_lib.doInvoke(core.clj:5953)
	at clojure.lang.RestFn.applyTo(RestFn.java:142)
	at clojure.core$apply.invokeStatic(core.clj:669)
	at clojure.core$load_libs.invokeStatic(core.clj:6016)
	at clojure.core$load_libs.doInvoke(core.clj:6000)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:669)
	at clojure.core$require.invokeStatic(core.clj:6038)
	at clojure.main$loading__6789__auto____9094.invoke(main.clj:11)
	at clojure.main__init.load(Unknown Source)
	at clojure.main__init.<clinit>(Unknown Source)
	... 55 more

If I add the dep explicitly, making my deps:

 :deps {org.clojure/clojure {:mvn/version "1.11.1"}
        org.clojure/spec.alpha {:mvn/version "0.3.218"}
        org.clojure/tools.cli {:mvn/version "1.0.206"}
        com.github.mwiede/jsch {:mvn/version "0.2.3"}
        nrepl/bencode {:mvn/version "1.1.0"}}

It still doesn't include it:

   Creating target/bbssh-0.1.0-SNAPSHOT-standalone.jar
  Including bbssh-0.1.0-SNAPSHOT.jar
  Including jsch-0.2.3.jar
  Including bencode-1.1.0.jar
  Including clojure-1.11.1.jar
  Including tools.cli-1.0.206.jar
  Including core.specs.alpha-0.2.62.jar
Done!

If I add both spec deps now, making deps:

      :deps {org.clojure/clojure {:mvn/version "1.11.1"}
        org.clojure/spec.alpha {:mvn/version "0.3.218"}
        org.clojure/core.specs.alpha {:mvn/version "0.2.62"}
        org.clojure/tools.cli {:mvn/version "1.0.206"}
        com.github.mwiede/jsch {:mvn/version "0.2.3"}
        nrepl/bencode {:mvn/version "1.1.0"}}

now both spec libs are included, but suddenly org.clojure/tools.cli is missing!

Creating target/bbssh-0.1.0-SNAPSHOT-standalone.jar
  Including bbssh-0.1.0-SNAPSHOT.jar
  Including jsch-0.2.3.jar
  Including bencode-1.1.0.jar
  Including clojure-1.11.1.jar
  Including core.specs.alpha-0.2.62.jar
  Including spec.alpha-0.3.218.jar
Done!

How does cambada determine the deps to include in the uberjar? Something very strange is happening here.

omit source files

Hi!

Nice tool. How to omit source files in uberjar?
Analog in Leiningen
;; Leave the contents of :source-paths out of jars (for AOT projects).
:omit-source true

Thank you.

AOT required for all jars

Currently there is no way to run the jar task without AOT compiling at least one namespace, or if there is it is not properly documented.

I suggest adding a --no-aot flag to allow a jar to be created which only copies source, and does not have any AOT compilation, since this is the normal method used for the creation of libraries.

Invalid sourceDirectory in generated pom

My deps.edn looks like this:

{
 :paths   ["src/cljc"]
 :aliases {
           :jar {:extra-deps {luchiniatwork/cambada {:mvn/version "1.0.2"}}
                 :main-opts  ["-m" "cambada.jar"
                              "-m" "xyz.core"
                              "--app-group-id" "abc"
                              "--app-artifact-id" "xyz"]}}}

Now clj -A:jar --app-version 0.0.1 generates pom.xml that includes this:

  <build>
    <sourceDirectory>src</sourceDirectory>
  </build>

while clj -Spom results with this:

  <build>
    <sourceDirectory>src/cljc</sourceDirectory>
  </build>

The jar built by cambada includes both cljs/xyz/core.cljc and xyz/core.cljc. I think that only the latter is correct and it might be result of bad sourceDirectory evaluation.

ArtifactNotFoundException for a pom artifact, when attempting to create uberjar

Hi there,

I'm trying to make an uberjar for a project that depends on metosin/spec-tools version 0.8.0, and I am getting the following error:

alex@deneb:/mnt/d/Projects/myproj$ clj -A:uberjar
Cleaning target
Creating target/classes
  Compiling myproj.core
Creating target/myproj-1.0.0-SNAPSHOT.jar
Updating pom.xml
Exception in thread "main" org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for com.fasterxml.jackson.core:jackson-databind:jar:2.9.7
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:323)
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:192)
        at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:253)
        at clojure.tools.deps.alpha.extensions.maven$eval2624$fn__2626.invoke(maven.clj:79)
        at clojure.lang.MultiFn.invoke(MultiFn.java:243)
        at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:170)
        at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:152)
        at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:215)
        at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:197)
        at cambada.uberjar$get_dep_jars.invokeStatic(uberjar.clj:109)
        at cambada.uberjar$get_dep_jars.invoke(uberjar.clj:107)
        at cambada.uberjar$apply_BANG_.invokeStatic(uberjar.clj:132)
        at cambada.uberjar$apply_BANG_.invoke(uberjar.clj:129)
        at cambada.cli$runner.invokeStatic(cli.clj:120)
        at cambada.cli$runner.invoke(cli.clj:115)
        at cambada.uberjar$_main.invokeStatic(uberjar.clj:142)
        at cambada.uberjar$_main.doInvoke(uberjar.clj:140)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:702)
        at clojure.core$apply.invokeStatic(core.clj:657)
        at clojure.main$main_opt.invokeStatic(main.clj:317)
        at clojure.main$main_opt.invoke(main.clj:313)
        at clojure.main$main.invokeStatic(main.clj:424)
        at clojure.main$main.doInvoke(main.clj:387)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:702)
        at clojure.main.main(main.java:37)
Caused by: org.apache.maven.model.resolution.UnresolvableModelException: Could not find artifact com.fasterxml.jackson:jackson-base:pom:2.9.7
        at org.apache.maven.repository.internal.DefaultModelResolver.resolveModel(DefaultModelResolver.java:178)
        at org.apache.maven.repository.internal.DefaultModelResolver.resolveModel(DefaultModelResolver.java:224)
        at org.apache.maven.model.building.DefaultModelBuilder.readParentExternally(DefaultModelBuilder.java:1051)
        at org.apache.maven.model.building.DefaultModelBuilder.readParent(DefaultModelBuilder.java:829)
        at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:331)
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:314)
        ... 26 more
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact com.fasterxml.jackson:jackson-base:pom:2.9.7
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:422)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:224)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:201)
        at org.apache.maven.repository.internal.DefaultModelResolver.resolveModel(DefaultModelResolver.java:174)
        ... 31 more
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact com.fasterxml.jackson:jackson-base:pom:2.9.7
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:412)
        ... 34 more

Why is it unable to find this pom artifact? I can see com.fasterxml.jackson:jackson-base:pom:2.9.7 in my ~/.m2 directory.

My deps.edn:

{:deps {metosin/spec-tools {:mvn/version "0.8.0"}}
 :aliases {:uberjar
           {:extra-deps
            {luchiniatwork/cambada {:mvn/version "1.0.0"}}
            :main-opts ["-m" "cambada.uberjar"
                        "-m" "myproj.core"]}}}

The dependency tree:

alex@deneb:/mnt/d/Projects/myproj$ clj -Stree
org.clojure/clojure 1.9.0
  org.clojure/core.specs.alpha 0.1.24
metosin/spec-tools 0.8.0
  org.clojure/spec.alpha 0.2.176
  com.fasterxml.jackson.core/jackson-databind 2.9.7
    com.fasterxml.jackson.core/jackson-annotations 2.9.0
    com.fasterxml.jackson.core/jackson-core 2.9.7

ArtifactDescriptorException: Failed to read artifact descriptor when running uberjar

I'm experiencing an exception

Exception in thread "main" org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.apache.httpcomponents:httpcore:jar:4.4.9
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:323)
...
	at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:215)
	at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:197)
	at cambada.uberjar$get_dep_jars.invokeStatic(uberjar.clj:109)
	at cambada.uberjar$get_dep_jars.invoke(uberjar.clj:107)
...

when running clj -A:uberjar -Srepro -Sforce on OS X High Sierra.

I've attached the simplest repro I could make for this.

This reproduces from a clean install:

empty ~/.m2 and ~/.clojure,
brew install clojure
run:

clj -Sdeps '{:deps
              {seancorfield/clj-new
                {:git/url "https://github.com/seancorfield/clj-new"
                 :sha "21ca1b27f46dc324be084ba839beca555aeda387"}}}' \
  -m clj-new.create \
  app \
  krukow/artifact-exception

Add to deps.edn

{:paths ["resources" "src"]
 :deps {org.clojure/clojure {:mvn/version "1.9.0"}
        clj-http {:mvn/version "3.8.0"}}
 :aliases
 {:uberjar
  {:extra-deps
   {luchiniatwork/cambada {:mvn/version "1.0.0"}}
   :main-opts ["-m" "cambada.uberjar"
               "-m" "krukow.artifact-exception"]}}
 }

However, if I add

:mvn/repos {"central" {:url "http://central.maven.org/maven2/"}
             "clojars" {:url "https://repo.clojars.org/"}}

things work as expected (I'm not interested in this since it's using http).

simple-repro.zip

Allow filtering of bad stuff (signatures etc) from input jars

Thanks for this, it is a nice tool.

I don't know if this is out of scope for your project, but when merging jars there are certain files which should probably be filtered out. For example, many jars have signatures in the META-INF folder which will fail to validate in the merged jar, causing Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes.

Similarly you may find files in META-INF/services which need concatenating.

boot has some waffle about this that might be helpful.

Error: Could not find or load main class

Hi! Thanks very much for putting this together. It's the most familiar and complete of the various deps based tools out there for packaging. I have been trying to get some of my lein based projects to use deps and am discovering some issues along the way.

After building the jar with the patch in #10 I ran into this error:

$ java -jar ./target/app-1.0.0-SNAPSHOT-standalone.jar
Error: Could not find or load main class app.main

I inspected the jar's contents and the class is indeed there. After some investigation discovered that if the jar contains cryptography libraries ie buddy and/or Bouncy Castle then there are various files present in META-INF/ with .SF, .RSA or .DSA suffixes which I believe are from signed jars. lein by default excludes these files when building uberjars.

Helpful Stackoverflow post: https://stackoverflow.com/questions/25842559/valid-jar-signature-for-javafx-projects/30922181

I'm interested in being a maintainer....

Cheers,
Jason Whitlark

clojure is always included in uberjar

While this is unlikely a problem for most, it is possible to build an uberjar with lein that does not include Clojure. The one use case I do have is with flyway. Vendor drivers that also depend on other libs can be uberjarred and droped in flyway's lib directory so that you can run migrations from the command line.

ERROR! Error: Main entry point class not found.

com.oracle.svm.core.util.UserError$UserException: Main entry point class 'cvfetcher.fetch-linkedin' not found.
        at com.oracle.svm.core.util.UserError.abort(UserError.java:65)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:260)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:448)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:113)
Error: Image build request failed with exit status 1

This happens when I try to compile a native image.

NoSuchFileException when performing uberjar with spec-alpha2 in deps.

Description

Attempting to build uberjar for a project that includes Clojure's unreleased spec-alpha2 as a git reference. It reaches a step that says "Including java" and throws an exception:

Cleaning target
Creating target/classes
Creating target/repro-case-1.0.0.jar
Updating pom.xml
Skipping paths: resources
Skipping coordinate: {:git/url https://github.com/clojure/spec-alpha2.git, :sha 2f84e3a37cab76d44c58785ff4481597429bc1d3}
Warning: The Main-Class specified does not exist within the jar. It may not be executable as expected. A gen-class directive may be missing in the namespace which contains the main method, or the namespace has not been AOT-compiled.
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Creating target/repro-case-1.0.0-standalone.jar
  Including repro-case-1.0.0.jar
  Including java
Execution error (NoSuchFileException) at sun.nio.fs.UnixException/translateToIOException (UnixException.java:92).
/root/.gitlibs/libs/org.clojure/spec-alpha2/2f84e3a37cab76d44c58785ff4481597429bc1d3/${project.basedir}/src/main/java

Full report at:
/tmp/clojure-17151214060258308065.edn

A minimal reproduction case is at https://gist.github.com/mtnygard/aaec1ec5ebb50c2405afd40bde40c5b9. That gist includes a deps.edn file that is sufficient to produce the error without any other sources required. The full report produced by cambada is included in that gist.

API change in tools.deps

Just wanted to drop a note that the newest version of tools.deps.alpha contains some api changes (moving towards getting out of alpha). So when you bump to tools.deps.alpha >= 0.9.745, your use of clojure.tools.deps.alpha.reader in cambda.cli needs to be updated - same function slurp-deps exists, but is now in clojure.tools.deps.alpha instead so just need to update the ns requires.

Feature request: lein-v (infer version numbers from git tags)

Cambada looks very cool, and I think it would help my team migrate from leiningen to tools.deps. One thing it's missing from our current workflow, though, is functionality like that of lein-v, which lets us specify what versions our projects are on using git tags.

I thought I'd check in and see if you had any thoughts on either including this functionality in cambada, or else accessing it through some sort of extension point if it was available as an external library. Essentially what I'd want is for the --app-version argument to be set from (some.ns/function), where some.ns is included on the classpath via my deps.edn alias.

I'd be happy to work on this if it seems like a good idea, let me know.

install and config deps.edn are not considered

When working with deps.edn, normally the files from the installation directory, from the config directory (usually ~/.clojure/deps.edn) and the local deps.edn are merged (using essentially a merge-with merge). Here only the local deps.edn file is considered.

Is this by choice or is it a bug?

resources folder is not included, but should be in uberjar.

Hello.

I want use cambada to build executable uberjar file.
This is my config
:uberjar {:extra-deps {luchiniatwork/cambada {:mvn/version "1.0.0"}} :main-opts ["-m" "cambada.uberjar" "-m" "my.app.core" "--no-copy-source"]}
I expect, that cambada will create uberjar file and resources folder will be included in it like leiningen does "lein uberjar". But in practice, cambada is ignoring resource folder and not include it in uberjar.

If I delete "--no-copy-source" parameter, then resources folder will be included in uberjar, but with source code (which is unwanted) and resource folder will be in wrong place.

Mike.

Cambada uberjar seems to expect a `user` namespace

Cleaning target
Creating target/classes
  Compiling user
Exception in thread "main" java.io.FileNotFoundException: Could not locate user__init.class, user.clj or user.cljc on classpath.

Exception in thread "main" java.lang.NoClassDefFoundError: clojure/data/xml/node/Element

This is such a cool project.. Thanks for working on this.. I am executing clj -R:cambada -m cambada.uberjar

Getting the following exception.

Cleaning target
Creating target/classes
  Compiling inject-github-id.core
Creating target/inject_github_id-1.0.0-SNAPSHOT.jar
Updating pom.xml
Exception in thread "main" java.lang.NoClassDefFoundError: clojure/data/xml/node/Element
	at cambada.jar$parse_xml$fn__2572.invoke(jar.clj:194)
	at clojure.core$filter$fn__5614.invoke(core.clj:2813)
	at clojure.lang.LazySeq.sval(LazySeq.java:40)
	at clojure.lang.LazySeq.seq(LazySeq.java:49)
	at clojure.lang.LazySeq.first(LazySeq.java:71)
	at clojure.lang.RT.first(RT.java:685)
	at clojure.core$first__5106.invokeStatic(core.clj:55)
	at clojure.core$first__5106.invoke(core.clj:55)
	at cambada.jar$parse_xml.invokeStatic(jar.clj:194)
	at cambada.jar$parse_xml.invoke(jar.clj:190)
	at cambada.jar$sync_pom$fn__2580.invoke(jar.clj:230)
	at cambada.jar$sync_pom.invokeStatic(jar.clj:227)
	at cambada.jar$sync_pom.invoke(jar.clj:222)
	at cambada.jar$apply_BANG_.invokeStatic(jar.clj:237)
	at cambada.jar$apply_BANG_.invoke(jar.clj:233)
	at cambada.uberjar$apply_BANG_.invokeStatic(uberjar.clj:130)
	at cambada.uberjar$apply_BANG_.invoke(uberjar.clj:129)
	at cambada.cli$runner.invokeStatic(cli.clj:120)
	at cambada.cli$runner.invoke(cli.clj:115)
	at cambada.uberjar$_main.invokeStatic(uberjar.clj:142)
	at cambada.uberjar$_main.doInvoke(uberjar.clj:140)
	at clojure.lang.RestFn.invoke(RestFn.java:397)
	at clojure.lang.AFn.applyToHelper(AFn.java:152)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at clojure.lang.Var.applyTo(Var.java:702)
	at clojure.core$apply.invokeStatic(core.clj:657)
	at clojure.main$main_opt.invokeStatic(main.clj:317)
	at clojure.main$main_opt.invoke(main.clj:313)
	at clojure.main$main.invokeStatic(main.clj:424)
	at clojure.main$main.doInvoke(main.clj:387)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:702)
	at clojure.main.main(main.java:37)
Caused by: java.lang.ClassNotFoundException: clojure.data.xml.node.Element
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:190)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:499)
	... 33 more

Problems with finding main entry class for uberjar and native-image

Got a simple cljfx project that i want to get built with cambada (ideally with native-image, but i'm kind of stuck on uberjar now).

Project repo: https://github.com/jumski/tone-collector/tree/add-cambada

My `deps.edn` file:

{:deps
 {org.clojure/clojure {:mvn/version "1.10.0"}
  cljfx {:mvn/version "1.6.7"}
  overtone/midi-clj {:mvn/version "0.5.0"}}
 :aliases
 {:uberjar {:extra-deps {luchiniatwork/cambada {:mvn/version "1.0.2"}}
            :main-opts ["-m" "cambada.uberjar"
                        "-m" "jumski.tone-collector.core"]}}}

Building `uberjar`

It mainly works with small warnings (that i assume are insignificant because they are about some logger)

warnings

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

But when i try to run the uberjar with

java -jar target/tone-collector-1.0.0-SNAPSHOT-standalone.jar

I got this error:

Error: Could not find or load main class jumski.tone-collector.core
Caused by: java.lang.ClassNotFoundException: jumski.tone-collector.core

By trial and error i figured out that when i provide the -m as a slash/underscore path like this, it will work:

-                        "-m" "jumski.tone-collector.core"]}}}
+                        "-m" "jumski/tone_collector/core"]}}}

I do not really understand what is the problem here.

Building `native-image`

When i try to use native-image support it fails with ERROR! Error: Main entry point class 'jumski.tone-collector.core not found for both jumski.tone-collector.core and jumski/tone_collector/core variants of main entry point spec.

Error for `jumski.tone-collector.core`

ERROR! Error: Main entry point class 'jumski.tone-collector.core' not found.
com.oracle.svm.core.util.UserError$UserException: Main entry point class 'jumski.tone-collector.core' not found.
        at com.oracle.svm.core.util.UserError.abort(UserError.java:68)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:300)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:501)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:115)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:528)
Error: Image build request failed with exit status 1

Error for `jumski/tone_collector/core`

ERROR! Error: Main entry point class 'jumski/tone_collector/core' not found.
com.oracle.svm.core.util.UserError$UserException: Main entry point class 'jumski/tone_collector/core' not found.
        at com.oracle.svm.core.util.UserError.abort(UserError.java:68)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:300)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:501)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:115)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:528)
Error: Image build request failed with exit status 1

Am i missing something or there is some kind or problem in cambada?

Versions

› clj --help
Version: 1.10.1.536

› gu --help
GraalVM Component Updater v2.0.0

› native-image --version
GraalVM Version 20.0.0 CE

› java --version
openjdk 11.0.6 2020-01-14
OpenJDK Runtime Environment GraalVM CE 20.0.0 (build 11.0.6+9-jvmci-20.0-b02)
OpenJDK 64-Bit Server VM GraalVM CE 20.0.0 (build 11.0.6+9-jvmci-20.0-b02, mixed mode, sharing)

native-image --graalvm-opts is wrong

I tried to pass the --enable-url-protocols=http but it failed miserably.

After some digging in the code it turns out that the implementation takes the opts and prepends them with the "-" character.

So I had to "hack" it by passing --graalvm-opt "-enable-url-protocols=http" so it works correctly. (note the absence of one "-" at the beginning of the option)

uberjar fails when using git libs

If deps.edn has a :git/url or :local/root such as the following then there is an exception thrown by ZipFile.open while generating the uberjar.

{:paths ["src" "resources"]
 :deps
  {commons   {:git/url "https://github.com/user/commons"
              :sha "caffa137fa1c865ee12f02ed100f11917e839c11"}}
 :aliases
 {:uberjar {:extra-deps
            {luchiniatwork/cambada {:mvn/version "1.0.0"}}
            :main-opts ["-m" "cambada.uberjar" "--main" "program.main"]}}

 ...
  Including httpcore-nio-4.4.6.jar
  Including component-0.3.2.jar
  Including clj
Exception in thread "main" java.io.FileNotFoundException: /home/user/.gitlibs/libs/commons/caffa137fa1c865ee12f02ed100f11917e839c11/src/clj (Is a directory)
	at java.util.zip.ZipFile.open(Native Method)
	at java.util.zip.ZipFile.<init>(ZipFile.java:225)
	at java.util.zip.ZipFile.<init>(ZipFile.java:155)
	at java.util.zip.ZipFile.<init>(ZipFile.java:169)
	at sun.reflect.GeneratedConstructorAccessor15.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at clojure.lang.Reflector.invokeConstructor(Reflector.java:180)
	at cambada.uberjar$include_dep.invokeStatic(uberjar.clj:103)
	at cambada.uberjar$include_dep.invoke(uberjar.clj:102)
	at clojure.core$partial$fn__5563.invoke(core.clj:2624)
	at clojure.core.protocols$naive_seq_reduce.invokeStatic(protocols.clj:62)
	at clojure.core.protocols$interface_or_naive_reduce.invokeStatic(protocols.clj:72)
	at clojure.core.protocols$fn__7852.invokeStatic(protocols.clj:169)
	at clojure.core.protocols$fn__7852.invoke(protocols.clj:124)
	at clojure.core.protocols$fn__7807$G__7802__7816.invoke(protocols.clj:19)
	at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
	at clojure.core.protocols$fn__7835.invokeStatic(protocols.clj:75)
	at clojure.core.protocols$fn__7835.invoke(protocols.clj:75)
	at clojure.core.protocols$fn__7781$G__7776__7794.invoke(protocols.clj:13)
	at clojure.core$reduce.invokeStatic(core.clj:6748)
	at clojure.core$reduce.invoke(core.clj:6730)
	at cambada.uberjar$write_components.invokeStatic(uberjar.clj:118)
	at cambada.uberjar$write_components.invoke(uberjar.clj:113)
	at cambada.uberjar$apply_BANG_.invokeStatic(uberjar.clj:138)
	at cambada.uberjar$apply_BANG_.invoke(uberjar.clj:129)
	at cambada.cli$runner.invokeStatic(cli.clj:120)
	at cambada.cli$runner.invoke(cli.clj:115)
	at cambada.uberjar$_main.invokeStatic(uberjar.clj:142)
	at cambada.uberjar$_main.doInvoke(uberjar.clj:140)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:702)
	at clojure.core$apply.invokeStatic(core.clj:657)
	at clojure.main$main_opt.invokeStatic(main.clj:317)
	at clojure.main$main_opt.invoke(main.clj:313)
	at clojure.main$main.invokeStatic(main.clj:424)
	at clojure.main$main.doInvoke(main.clj:387)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:702)
	at clojure.main.main(main.java:37)

Command line option --out does not work correctly

When --out tmp option used classes compiled to default target/classes not into tmp/classes.

SAXParseException when buliding uberjar

I'm getting the following stacktrace when I attempt to build an uberjar:

[Fatal Error] :1:628: The content of elements must consist of well-formed character data or markup.
ERROR:  'The content of elements must consist of well-formed character data or markup.'
Exception in thread "main" javax.xml.transform.TransformerException: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 628; The content of elements must consist of well-formed character data or markup.
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:740)
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:343)
        at clojure.data.xml.jvm.pprint$indent_xml.invokeStatic(pprint.clj:27)
        at clojure.data.xml.jvm.pprint$indent_xml.invoke(pprint.clj:23)
        at clojure.data.xml$indent.invokeStatic(xml.clj:158)
        at clojure.data.xml$indent.doInvoke(xml.clj:153)
        at clojure.lang.RestFn.invoke(RestFn.java:425)
        at clojure.lang.AFn.applyToHelper(AFn.java:156)
        at clojure.lang.RestFn.applyTo(RestFn.java:132)
        at clojure.core$apply.invokeStatic(core.clj:661)
        at clojure.core$apply.invoke(core.clj:652)
        at clojure.data.xml$indent_str.invokeStatic(xml.clj:164)
        at clojure.data.xml$indent_str.doInvoke(xml.clj:160)
        at clojure.lang.RestFn.invoke(RestFn.java:410)
        at clojure.tools.deps.alpha.gen.pom$sync_pom.invokeStatic(pom.clj:135)
        at clojure.tools.deps.alpha.gen.pom$sync_pom.invoke(pom.clj:123)
        at cambada.jar$sync_pom.invokeStatic(jar.clj:225)
        at cambada.jar$sync_pom.invoke(jar.clj:222)
        at cambada.jar$apply_BANG_.invokeStatic(jar.clj:237)
        at cambada.jar$apply_BANG_.invoke(jar.clj:233)
        at cambada.uberjar$apply_BANG_.invokeStatic(uberjar.clj:130)
        at cambada.uberjar$apply_BANG_.invoke(uberjar.clj:129)
        at cambada.cli$runner.invokeStatic(cli.clj:120)
        at cambada.cli$runner.invoke(cli.clj:115)
        at cambada.uberjar$_main.invokeStatic(uberjar.clj:142)
        at cambada.uberjar$_main.doInvoke(uberjar.clj:140)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:702)
        at clojure.core$apply.invokeStatic(core.clj:657)
        at clojure.main$main_opt.invokeStatic(main.clj:317)
        at clojure.main$main_opt.invoke(main.clj:313)
        at clojure.main$main.invokeStatic(main.clj:424)
        at clojure.main$main.doInvoke(main.clj:387)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:702)
        at clojure.main.main(main.java:37)
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 628; The content of elements must consist of well-formed character data or markup.
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1239)
        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transformIdentity(TransformerImpl.java:632)
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:728)
        ... 35 more
---------
org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 628; The content of elements must consist of well-formed character data or markup.
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1239)
        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transformIdentity(TransformerImpl.java:632)
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:728)
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:343)
        at clojure.data.xml.jvm.pprint$indent_xml.invokeStatic(pprint.clj:27)
        at clojure.data.xml.jvm.pprint$indent_xml.invoke(pprint.clj:23)
        at clojure.data.xml$indent.invokeStatic(xml.clj:158)
        at clojure.data.xml$indent.doInvoke(xml.clj:153)
        at clojure.lang.RestFn.invoke(RestFn.java:425)
        at clojure.lang.AFn.applyToHelper(AFn.java:156)
        at clojure.lang.RestFn.applyTo(RestFn.java:132)
        at clojure.core$apply.invokeStatic(core.clj:661)
        at clojure.core$apply.invoke(core.clj:652)
        at clojure.data.xml$indent_str.invokeStatic(xml.clj:164)
        at clojure.data.xml$indent_str.doInvoke(xml.clj:160)
        at clojure.lang.RestFn.invoke(RestFn.java:410)
        at clojure.tools.deps.alpha.gen.pom$sync_pom.invokeStatic(pom.clj:135)
        at clojure.tools.deps.alpha.gen.pom$sync_pom.invoke(pom.clj:123)
        at cambada.jar$sync_pom.invokeStatic(jar.clj:225)
        at cambada.jar$sync_pom.invoke(jar.clj:222)
        at cambada.jar$apply_BANG_.invokeStatic(jar.clj:237)
        at cambada.jar$apply_BANG_.invoke(jar.clj:233)
        at cambada.uberjar$apply_BANG_.invokeStatic(uberjar.clj:130)
        at cambada.uberjar$apply_BANG_.invoke(uberjar.clj:129)
        at cambada.cli$runner.invokeStatic(cli.clj:120)
        at cambada.cli$runner.invoke(cli.clj:115)
        at cambada.uberjar$_main.invokeStatic(uberjar.clj:142)
        at cambada.uberjar$_main.doInvoke(uberjar.clj:140)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:702)
        at clojure.core$apply.invokeStatic(core.clj:657)
        at clojure.main$main_opt.invokeStatic(main.clj:317)
        at clojure.main$main_opt.invoke(main.clj:313)
        at clojure.main$main.invokeStatic(main.clj:424)
        at clojure.main$main.doInvoke(main.clj:387)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:702)
        at clojure.main.main(main.java:37)

Starts running main function

When I'm trying to make an uberjar, the server I'm building starts running!

luchiniatwork / cambada Goto Github PK

cambada's People

Contributors

Stargazers

Watchers

Forkers

cambada's Issues

Description

My deps.edn file:

Building uberjar

Building native-image

Versions

Recommend Projects

Recommend Topics

Recommend Org

My `deps.edn` file:

Building `uberjar`

Building `native-image`