Resource leak about babashka.curl HOT 10 CLOSED

Thank you. Released: https://github.com/borkdude/babashka/releases/tag/v0.0.86

from babashka.curl.

@lukaszkorecki Maybe parsing headers via a temporary file isn't the best idea ever. This was done because when parsing the headers from the same stream as the response with the --include option, there is no reliable way to detect when the headers end and the response body starts.
Could it be that we have to call .destroy on the curl process to free up these resources?

from babashka.curl.

I can reproduce the problem locally with this script:

(require '[babashka.curl :as curl])

(while true
  (Thread/sleep 1000)
  (let [resp (curl/get "https://www.clojure.org")]
    (:body resp)
    (.destroy (:process resp))) ;; this is not helping
)

Note that this script doesn't exhibit the problem:

(while true
  (Thread/sleep 1000)
  (let [f (java.io.File/createTempFile "babashka.curl" ".headers")]
    (spit f "foo")
    (.delete f)))

from babashka.curl.

Yeah, the process included in the response map was my first hint - and I did try the .destroy method on it and it didn't help. I wonder if it needs to be called before deleting the tempfile?

from babashka.curl.

@lukaszkorecki I'll do some testing...

Meanwhile you can poll your endpoint using something like this:

(clojure.java.shell/sh "curl" "https://www.clojure.org")))

or use HttpURLConnection as follows:

https://github.com/borkdude/deps.clj/blob/64589845e014191b33d97cb1983dca504ba1dff2/src/borkdude/deps.clj#L188.

The library https://github.com/borkdude/clj-http-lite is based around HttpURLConnection and this library can also be used with babashka.

from babashka.curl.

So far I've found that running with the JVM (instead of GraalVM) has the same behavior so at least it's not a GraalVM problem.

from babashka.curl.

@lukaszkorecki Found the culprit!

(defn- read-headers [^File header-file]
  (line-seq (io/reader header-file)))

This leaves the file handle open. Will fix ASAP.

from babashka.curl.

@borkdude oh, nice! Thank you 🎉

from babashka.curl.

@lukaszkorecki Will publish a new version of bb. Thanks for finding this.

from babashka.curl.

@borkdude Thank you for creating BB - finally it feels like we have a viable Clojure for small tasks :-)

from babashka.curl.