ring-clojure / ring Goto Github PK
View Code? Open in Web Editor NEWClojure HTTP server abstraction
License: MIT License
Clojure HTTP server abstraction
License: MIT License
Hi,
When given params map with a native clojure array behind one of the keys, wrap-nested-params corrupts it.
Request (POST body):
{"Platforms": ["Android", "iOS"]}
Parsed request (with ring-json/wrap-params):
{...
:json-params {"Platforms" ["Android" "iOS"]}
:params {"Platforms" ["Android" "iOS"]}
...}
After ring.middleware.params/wrap-params:
{...
:json-params {"Platforms" ["Android" "iOS"]}
:params {"Platforms" ["Android" "iOS"]}
...}
After ring.middleware.nested-params/wrap-nested-params:
{...
:json-params {"Platforms" ["Android" "iOS"]}
:params {"Platforms" "iOS"}
...}
After ring.middleware.keyword-params/wrap-keyword-params:
{...
:json-params {"Platforms" ["Android" "iOS"]}
:params {:Platforms "iOS"}
...}
Is it desired behaviour or not? Or is it some kind of interference with ring-json?
P.S. Why I've used those 3 wrappers? They're part of compojure.handler/api.
I came across this by chance when I mistyped the name of an input field. Here's my set of tests, all of them provide the expected response except for the last one.
The order is important to generate this error. All the tests below pass unless you create the value as a string first (passing in val before any val[]'s). I can provide more information if necessary, using ring 1.1.8.
These return the expected results
localhost/url/?val=three
{:val "three"}
localhost/url/?val[]=one&val[]=two
{:val ["one" "two"]}
localhost/url/?val=one&val=two&val=three
{:val "three"}
localhost/url/?val[]=one&val[]=two&val=three
{:val "three"}
This one throws an error
localhost/url/?val=three&val[]=one&val[]=two
Server error
Stacktrace
java.lang.ClassCastException: java.lang.String cannot be cast to clojure.lang.IPersistentCollection
at clojure.core$conj.invoke(core.clj:83)
at ring.middleware.nested_params$assoc_nested.invoke(nested_params.clj:23)
at ring.middleware.nested_params$assoc_nested.invoke(nested_params.clj:22)
at ring.middleware.nested_params$nest_params$fn__824.invoke(nested_params.clj:44)
at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:58)
at clojure.core.protocols$fn__6041.invoke(protocols.clj:98)
at clojure.core.protocols$fn__6005$G__6000__6014.invoke(protocols.clj:19)
at clojure.core.protocols$seq_reduce.invoke(protocols.clj:31)
at clojure.core.protocols$fn__6026.invoke(protocols.clj:54)
at clojure.core.protocols$fn__5979$G__5974__5992.invoke(protocols.clj:13)
at clojure.core$reduce.invoke(core.clj:6177)
at ring.middleware.nested_params$nest_params.invoke(nested_params.clj:46)
at clojure.lang.AFn.applyToHelper(AFn.java:163)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:619)
at clojure.core$update_in.doInvoke(core.clj:5587)
at clojure.lang.RestFn.invoke(RestFn.java:467)
at ring.middleware.nested_params$wrap_nested_params$fn__830.invoke(nested_params.clj:64)
at ring.middleware.params$wrap_params$fn__749.invoke(params.clj:55)
at ring.adapter.jetty$proxy_handler$fn__1310.invoke(jetty.clj:18)
at ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$0.handle(Unknown Source)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
at org.eclipse.jetty.server.Server.handle(Server.java:349)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:452)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:884)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:938)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:76)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
at java.lang.Thread.run(Thread.java:679)
When a cookie-attrs :path is specified and no session :root is specified, the default session :root "/" replaces the specified cookie-attrs :path.
(wrap-session {:cookie-attrs {:path "/mypath"}})
produces a cookie with a path of "/" rather than "/mypath".
(defn wrap-session [options]
"snipped code from wrap-session"
(let [session-root (options :root "/")
cookie-attrs (merge (options :cookie-attrs) {:path session-root})]
(println (str "cookie-attrs: " cookie-attrs))))
ring.middleware.keyword-params has a fixed regex for what it considers legal keywords and there is no way to configure or override that.
I want to have params that include "." dots as keywords, but it seems that I need to copy the rest of this file to accomplish that.
clj-stacktrace 2.5.0, the version Ring 1.2.0 uses, has a bug that raises an IndexOutOfBoundsException
for exceptions with long messages, so if you use wrap-stacktrace
some exceptions will be seen incorrectly. See this:
mmcgrana/clj-stacktrace#20
mmcgrana/clj-stacktrace#20 (comment)
Don't know if this bug is still present in clj-stacktrace 2.6.0.
The current behaviour of the jetty adapter is to set charset=UTF-8 on every response. It turns out that doing this on image responses when HTTPS is being used causes IE7/8 (I think 9 as well, though didn't verify) to sporadically not display images (or very consistently in the case of IE7).
My temporary solution to this was to disable the charset in ring-jetty-adapter and instead use the following middleware:
(defn wrap-utf8
"Wrap UTF-8 encoding only on text responses."
[handler]
(fn [req]
(let [resp (handler req)
ctype (get-in resp [:headers "Content-Type"])]
(if (and (> (count ctype) 5) (= (subs ctype 0 5) "text/" ))
(charset resp "UTF-8")
resp))))
The spec contains this right now:
The HTTP request method, must be one of :get, :head, :options, :put, :post, or :delete.
HTTP 1.1 allows extensions (note "extension-method"). WebDAV has been around since 1999, and it has a lot of them.
The PATCH method is very popular now, it's the primary method for updates in Rails and it's used in the GitHub API.
I see no reason why Ring should only allow the standard HTTP methods.
On Windows, the resource and static file handling (.js .css files) in the development server is serving files that have not changed relative the the browsers If-Modified-Since date.
Headers:
GET /js/jquery.iframe-transport.js HTTP/1.1
Host: localhost:3000
Connection: keep-alive
Cache-Control: max-age=0
Accept: */*
If-Modified-Since: Thu, 02 May 2013 19:40:39 +0000
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Referer: http://localhost:3000/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
HTTP/1.1 200 OK
Date: Wed, 15 May 2013 15:48:46 GMT
Content-Length: 8902
Last-Modified: Thu, 02 May 2013 19:40:39 +0000
Content-Type: text/javascript
Server: Jetty(7.6.1.v20120215)
The term stacktrace could be colored using the pst+
function in clj-stacktrace, and some work could be done to improve the HTML version as well.
a reference to assoc-conj can be found in ring-core/src/ring/middleware/multipart_params.clj in 1.2.0-beta1
In a project I happened to have both and index.html and index.js file, and find-index-file always returned the javascript. Perhaps this should be more configurable, or prefer html over anything else.
In ring-core 1.2.1, when compiling with lein check, or running a repl in lein, I'm getting this explosion:
Exception in thread "main" java.lang.NoClassDefFoundError: javax/servlet/http/HttpServletRequest, compiling:(ring/middleware/multipart_params.clj:39:5)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6567)
at clojure.lang.Compiler.analyze(Compiler.java:6361)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6548)
at clojure.lang.Compiler.analyze(Compiler.java:6361)
at clojure.lang.Compiler.analyze(Compiler.java:6322)
at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3624)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6562)
at clojure.lang.Compiler.analyze(Compiler.java:6361)
at clojure.lang.Compiler.analyze(Compiler.java:6322)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5708)
at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5139)
at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3751)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6558)
at clojure.lang.Compiler.analyze(Compiler.java:6361)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6548)
at clojure.lang.Compiler.analyze(Compiler.java:6361)
at clojure.lang.Compiler.access$100(Compiler.java:37)
at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:529)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6560)
at clojure.lang.Compiler.analyze(Compiler.java:6361)
at clojure.lang.Compiler.analyze(Compiler.java:6322)
at clojure.lang.Compiler.eval(Compiler.java:6623)
at clojure.lang.Compiler.load(Compiler.java:7064)
at clojure.lang.RT.loadResourceScript(RT.java:370)
at clojure.lang.RT.loadResourceScript(RT.java:361)
at clojure.lang.RT.load(RT.java:440)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5018.invoke(core.clj:5530)
at clojure.core$load.doInvoke(core.clj:5529)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5336)
at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
at clojure.core$load_lib.doInvoke(core.clj:5374)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:619)
at clojure.core$load_libs.doInvoke(core.clj:5417)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:621)
at clojure.core$use.doInvoke(core.clj:5507)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at compojure.handler$eval11534$loading__4910__auto____11535.invoke(handler.clj:1)
at compojure.handler$eval11534.invoke(handler.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:6619)
at clojure.lang.Compiler.eval(Compiler.java:6608)
at clojure.lang.Compiler.load(Compiler.java:7064)
at clojure.lang.RT.loadResourceScript(RT.java:370)
at clojure.lang.RT.loadResourceScript(RT.java:361)
at clojure.lang.RT.load(RT.java:440)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5018.invoke(core.clj:5530)
at clojure.core$load.doInvoke(core.clj:5529)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5336)
at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
at clojure.core$load_lib.doInvoke(core.clj:5374)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:619)
at clojure.core$load_libs.doInvoke(core.clj:5413)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:619)
at clojure.core$require.doInvoke(core.clj:5496)
at clojure.lang.RestFn.invoke(RestFn.java:3894)
The stack starts at the point in my code where I wrap the app in:
(->> app
.. stuff...
ring.middleware.multipart-params/wrap-multipart-params
... more stuff ...
)
What's baffling to me is that the ring-core line cited in the above stacktrace is the last line of this function:
(defn- file-item-seq
"Create a seq of FileItem instances from a request context."
[context]
(file-item-iterator-seq
(.getItemIterator (FileUpload.) context)))
What that has to do with HttpServletRequest is utterly opaque to me.
Sadly, I haven't been able to isolate it any further, because I have very little understanding of the underlying Java stuff, and no idea what a HttpServletRequest is, why it's in ring, or why it could be exploding.
FWIW, the server I'm using here is aleph, not jetty. If this is a problem with aleph, please let me know and I'll ask Zach about it instead
Still has http://github.com/mmcgrana/ring
A HEAD request to a static file or resource should return the same headers and response code as a GET request.
(V 1.1.0, Clojure 1.3)
Here's a minimal failing test case:
(percent-decode "%24")
or
(percent-decode (percent-encode "$"))
And here's a monkey patch to demonstrate a fix:
(with-redefs [ring.util.codec/double-escape (ƒ [x] (.replace (.replace x "\\" "\\\\") "$" "\\$")) ] (percent-decode "%24"))
Exception from calling url-decode:
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: 3
at java.lang.String.charAt(String.java:686)
at java.util.regex.Matcher.appendReplacement(Matcher.java:711)
at clojure.string$replace_by.invoke(string.clj:58)
at clojure.string$replace.invoke(string.clj:82)
at ring.util.codec$percent_decode.doInvoke(codec.clj:31)
at clojure.lang.RestFn.invoke(RestFn.java:423)
at ring.util.codec$url_decode.doInvoke(codec.clj:51)
at clojure.lang.RestFn.invoke(RestFn.java:410)
Thanks to Chris Hapgood with whom I worked to figure this out.
I'm not sure if you will consider that as a bug, but in tests you submit a GET
request with body, and then check that body is echoed by a handler.
Jetty passes body to your handler, so that works. But Undertow does not and I've spent quite a bit of time trying to understand what's going on until I noticed it's a get. :-) So while it works, maybe it's worth changing it to http/post
, what do you think?
Implementation of temp-file-store
in the ring.middleware.multipart-params.temp-file
creates too many futures objects and this leads to too many threads running on web services that handles many multipart uploads...
P.S. I'll try to refactor code to use single expiration thread or something like...
P.P.S. Initial discussion was at mmcgrana#74
Nested JavaScript arrays in AJAX call data to a ring application fail to parse. I think I can reproduce the error as far down as form-decode
.
Disclaimer: I'm new to Ring and web programming in general. Please correct me if this ticket should be filed elsewhere.
$( document ).ready( function() {
$.ajax({
url: "/query/",
data: {want: "C", constraints: [["A", "B"]]},
success: function(data, aStatus, dummy){
alert(data);
}
});
});
{:scheme :http, :request-method :get,
:query-string want=C&constraints%5B0%5D%5B%5D=A&constraints%5B0%5D%5B%5D=B,
:route-params {}, :content-type nil, :params {},
:body #<HttpInput org.eclipse.jetty.server.HttpInput@1bd06bf>
....}
(clojure.pprint/pprint (-> result :query-string form-decode))
{"constraints[0][]" ["A" "B"], "want" "C"}
{"constraints" [["A" "B"]], "want" "C"}
wrap-multipart-params
is different from other wrap-* functions
(defroutes all-routes
; start a thread per run
(POST "/" [] (wrap-multipart-params handler))
)
temp-file-store
start a thread when called: https://github.com/ring-clojure/ring/blob/master/ring-core/src/ring/middleware/multipart_params/temp_file.clj#L39
ring-core depends on [org.clojure/tools.reader "0.7.3"]
, while clojurescript (starting with -1909 I think) depends on tools.reader 0.7.8. In some configurations the ring-core dependency overrides the other, which produces hard-to-track-down error messages in ClojureScript.
Excluding the ring-core dependency and using 0.7.8 instead doesn't seem to cause problems in my project. If no tests fail I'd like to suggest updating the dependency on tools.reader to 0.7.8 to play fine with ClojureScript again.
Please see
http://stackoverflow.com/questions/14309400/a-directory-explorer-middleware-for-ring-clojure
I find that listing directories like apache does is something super useful to me when debugging and exploring my projects.
A content-type header with a value like "vnd.example-com.foo+json; version=1.0" will lose the additional parameter when it passes through ring.util.servlet/set-content-type
When using wrap-nested-params , parameters like:
http://someurl?param=value1¶m=value2
are mapped to
{:params {:param "value2"}} instead of {:params {:param [ "value1" "value2"]}} in the request map.
I understand why it does not do the early-exit method of comparing two strings, but instead a constant time algorithm, to avoid a timing attack. That is good.
However, I don't think it is implemented correctly. By calling String's .getBytes method, two strings that have equal length could each be transformed into a byte array with different lengths from each other, if they have different data. For example:
user=> (defn utf-16-code-units-to-str [code-units]
(apply str (map char code-units)))
user=> (def s1 (utf-16-code-units-to-str [65 66 67 6500]))
#'user/s1
user=> (def s2 (utf-16-code-units-to-str [65 66 67 100]))
#'user/s2
user=> (.length s1)
4
user=> (.length s2)
4
user=> (seq (.getBytes s1))
(65 66 67 -31 -91 -92)
user=> (seq (.getBytes s2))
(65 66 67 100)
user=> (map bit-xor (.getBytes s1) (.getBytes s2))
(0 0 0 -123)
Thus secure-compare could return true for two different strings, as long as the arrays returned by .getBytes for the two were equal in the elements of the shorter one.
I would recommend replacing the two calls to (.getBytes) there with (map int s1) and (map int s2). Those will always return sequences containing the same number of integers as the original strings have Java characters.
In addition, there seem to be several calls to .getBytes throughout ring that do not specify a character encoding. This could lead to different behavior on different systems with different default character encodings. Not sure if that is desired or not, but it seems like it could lead to a lack of portability if not bugs.
In the 1.2.0 documentation, the doctoring for ring.util.response.set-cookies? says it "Requires the handler to be wrapped in the wrap-cookies middleware" [here: http://ring-clojure.github.io/ring/ring.util.response.html#var-set-cookie ], but there is no namespace for ring.middleware.cookies or ring.middleware.wrap-cookies or anything else that seems relevant.
Not trying to pick nits. I'm new to ring and am genuinely confused about it's cookie-handling.
Hi James,
I think I've just come across a problem with the wrap-params
middleware.
A minimal reproduction:
(ns ring-bug
(:require [compojure.handler]
[compojure.core :as comp :refer (defroutes routes)]
[ring.adapter.jetty :as jetty]))
(defroutes my-app
;;(compojure.handler/api (comp/GET "/foo" {} "foo"))
(compojure.handler/site (comp/POST "/bar" {p :params} (str "Params: " p))))
(defonce server
(jetty/run-jetty #'my-app {:join? false :port 8086}))
;; When /api is commented out:
;; curl -d "name=steve" http://localhost:8086/bar => Params: {:name "steve"}
;; When /api is uncommented:
;; curl -d "name=steve" http://localhost:8086/bar => Params: {}
It looks to me like wrap-params
is mutating something in place (perhaps the org.eclipse.jetty.server.HttpInput?), which breaks the second call to wrap-params
.
Any ideas?
Thanks!
why is stacktrace throwing this error on a wrong route, how should it be handled
java.lang.IndexOutOfBoundsException
RT.java:784 clojure.lang.RT.nthFrom
RT.java:753 clojure.lang.RT.nth
utils.clj:19 clj-stacktrace.utils/quartile1
utils.clj:36 clj-stacktrace.utils/fence
repl.clj:95 clj-stacktrace.repl/find-source-width
repl.clj:107 clj-stacktrace.repl/pst-on
repl.clj:123 clj-stacktrace.repl/pst-str
RestFn.java:408 clojure.lang.RestFn.invoke
stacktrace.clj:17 ring.middleware.stacktrace/wrap-stacktrace-log[fn]
stacktrace.clj:79 ring.middleware.stacktrace/wrap-stacktrace-web[fn]
jetty.clj:18 ring.adapter.jetty/proxy-handler[fn](Unknown Source) ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$0.handle
HandlerWrapper.java:111 org.eclipse.jetty.server.handler.HandlerWrapper.handle
Server.java:349 org.eclipse.jetty.server.Server.handle
AbstractHttpConnection.java:452 org.eclipse.jetty.server.AbstractHttpConnection.handleRequest
AbstractHttpConnection.java:884 org.eclipse.jetty.server.AbstractHttpConnection.headerComplete
AbstractHttpConnection.java:938 org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete
HttpParser.java:634 org.eclipse.jetty.http.HttpParser.parseNext
HttpParser.java:230 org.eclipse.jetty.http.HttpParser.parseAvailable
AsyncHttpConnection.java:76 org.eclipse.jetty.server.AsyncHttpConnection.handle
SelectChannelEndPoint.java:609 org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle
SelectChannelEndPoint.java:45 org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run
QueuedThreadPool.java:599 org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
QueuedThreadPool.java:534 org.eclipse.jetty.util.thread.QueuedThreadPool$3.run
Thread.java:679 java.lang.Thread.run
I recently got bit by the fact that :body
is an input string. Here was what happened:
(def site-routes
(-> (routes about-routes
legal-routes
paste-routes
user-routes
home-routes
login-routes)
(api)
(wrap-noir-flash)
(wrap-noir-session {:store (monger-store "sessions")})
(wrap-strip-trailing-slash)
(wrap-prod-middleware)))
(def a-routes
(-> api-routes
(api)
(wrap-noir-session {:store (monger-store "sessions")})
(wrap-strip-trailing-slash)
(wrap-prod-middleware)))
(defroutes handler
site-routes
a-routes
(resources "/")
(not-found (four-zero-four)))
This is denser than necessary, but allow me to explain.
I have two sets of routes that are both wrapped with api
and thus wrap-params
. I noticed that my a-routes
never had any :form-params
. I spent a while trying to debug this and figure out why with @brehaut and we came to the conclusion that the wrap-params
in site-routes
was reading the :body
of the request, so a-routes
's wrap-params
was unable to do so.
I'm not sure these read-once semantics make sense. @brehaut was proposing that it could instead be like a lazy cons where the tail is either pending or realized, or there could perhaps be a middleware for that. At the very least, this should be documented. It feels very wrong to be bit by mutability in a functional framework!
ring.middleware.stacktrace/wrap-stacktrace-web
and wrap-stacktrace-log
wraps the handler inside a (try handler (catch Exception ex ..)
block. But if the handler contains assert
expressions and some of they fail, these exceptions will not be catched by Ring, because assert
throws AssertionError
's and they aren't derived from java.lang.Exception
. So nothing will be printed to the log, and Ring will return an empty 500 Server Error.
Here is a test:
;; http://localhost:8081/zerodiv will show a backtrace
;; http://localhost:8081/assert will not show it
(def my-handler
(->
#(case (:uri %)
"/assert" (assert false)
"/zerodiv" (/ 1 0))
ring.middleware.stacktrace/wrap-stacktrace
))
(ring.adapter.jetty/run-jetty #'my-handler {:port 8081
:join? false})
I think wrap-stracktrace-*
must be modified to catch Throwable
or maybe add an (catch AssertionError
block if you don't want to catch all the Throwable~
s/Error
's exceptions.
I use wrap-session to manage cookie-based sessions for my users and many times when running unit tests I noticed that a certain test would stall for over 10s. I narrowed it down to one specific situation:
I make a request to the ring app. The session map is empty as there is no ring-session= cookie associated with it. Say a new :session map is associated with the response. wrap-response will occasionally (not always, maybe once every 3-5 test runs) will stall for 10+s. The other times there is no delay.
(-> routes
[...]
(print-response "above wrap session")
(wrap-session {:store (cookie-store {:key (env :cookie-key)})
:cookie-attrs {:domain (env :cookie-domain)
:max-age 31536000
:secure true
:http-only true}})
(print-response "below wrap session")
[...])
Here's the output of the above:
2013-Jul-19 13:38:19 -0700 alex-haswell DEBUG [classroom.middleware] - above wrap session response: {:session {:teacher-id 1}, :status 200, :headers {}, :body {}}
2013-Jul-19 13:38:33 -0700 alex-haswell DEBUG [classroom.middleware] - below wrap session response: {:status 200, :headers {"Set-Cookie" ("ring-session=IkPHd4p0KNxCQj3o%2BqkXLoqQOlF%2BDOpYm%2FZGd2zo6BY%3D--JNyHj01WUANP5kDrFDJ5k6gqj2Hsemzy4CbhHP9suHA%3D;HttpOnly;Secure;Max-Age=31536000;Domain=localhost.com;Path=/")}, :body {}}
If you look at the timbre timestamps, you can see operation took a considerable while to complete. I'm wondering if I'm doing something wrong here that's causing the slow-down. Is this by design?
I'm using Ring through Compojure 1.1.5, with the lein-ring 0.8.2 plugin. My handler is set up to look something like this:
(defroutes app-routes
(GET "/info/:item" [item](-> %28info/get-data item endpoint%29
u/json-response))
(GET "/" [](-> %28io/resource)
(response/render {"Content-Type" "text/html"})))
(route/resources "/")
(route/not-found "Not Found"))
(def app
(handler/site app-routes))
The problem is, when I call 'lein ring uberwar', I get a WAR file with the index, js, and related front-end files stuck way back in WEB-INF/classes/public. Which isn't what I expect compared to java-generated WAR files, where those front-end resources are put in the root of the WAR.
This doesn't cause an issue with locally-run Tomcats, but deploying to Amazon's elastic beanstalk results in my index.html file being recognized as text/plain. So I end up having to explicitly call the service at http://what.ever/index.html, which is kind of hokey, right?
This might be an issue better aimed at lein-ring, but I thought I'd start here first.
(the project I'm seeing this problem with is sourced at http://github.com/ryankohl/nasa-data-browser)
(also posted to http-kit as I'm not sure what is the right "layer" to solve this problem)
This isn't so much a "here's a bug in the code" but more of a "is it possible to do X with ring."
I'm using
{:client-side [clojurescript]
:server-side [clojure, ring, http-kit]}
Now, I want my code to be "robust" vs dropped connections (phone / wifi that often times out.)
Is there a way to simulate this with ring middleware?
Basically, I need it to do things like:
(1) arbitrarily ignore requests
(2) arbitrarily delay requests
(3) arbitrarily kill web-sockets / comet / long-polling connections
I'd like this to be done in an as "unclean" way as possible -- i.e. instead of calling close on a websocket, have it abruptly killed / time out.
Thanks!
(PS, if this question doesn't belong here / should be posted in a forum somewhere, please let me know.)
wrap-resource doesn't make an attempt to set any response headers, such as last-modified and content-length; and wrap-file-info isn't compatible with wrap-resource unless the resource came from a file on the class-path. If the resource came from a jar file, we end up without the response headers set.
I package my app as an uber-jar, so I'd like wrap-resource to support conditional gets.
It is easy enough to get the content-length and last-modified date for a resource URL, using something like the following. This works whether the resource comes from a file or a jar file (or anywhere else for that matter)
(defn last-modified-and-content-length
[^java.net.URL res-url]
(let [ucon (.openConnection res-url)]
[(.getLastModified ucon) (.getContentLengthLong ucon)]))
Could we incorporate something like this into wrap-resource?
The 1.1.6 changelog says "Removed default charset being incorrectly set on images," however, this is still happening.
Code to reproduce:
(ns testcase.core
(:require [ring.util.response :as resp]))
(defn handler [request]
(-> (resp/response "")
(resp/header "Content-Type" "image/png")))
project.clj:
(defproject testcase "1.0.0-SNAPSHOT"
:description "FIXME: write description"
:dependencies [[org.clojure/clojure "1.3.0"]
[ring/ring-jetty-adapter "1.1.6"]
[ring/ring-core "1.1.6"]
[ring/ring-servlet "1.1.6"]]
:dev-dependencies [[lein-ring "0.7.5"]]
:ring {:handler testcase.core/handler})
I ran this with lein ring server
and executed:
$ curl -i localhost:3000
HTTP/1.1 200 OK
Date: Wed, 26 Sep 2012 17:02:11 GMT
Content-Type: image/png;charset=ISO-8859-1
Content-Length: 0
Server: Jetty(7.6.1.v20120215)
$
It's also set for many other types it shouldn't be, including application/pdf
, application/octet-stream
, application/x-gzip
, etc.
While ring spec says that header names in request from ring adapter are lower-cased it says nothing about headers returned in responses. Actually response headers are usually written Camel-Cased. This is a problem for middlewares that may wish to post-process response headers after a handler.
For example code that reads and writes Last-Modified header should account for header name differences in request and response. Other use case would be a middleware that sets header only if it is not already present - if character cases are not considered header might be set twice.
See https://github.com/PetrGlad/ring/commit/get-header/ for a hack that fixes comparison in not-modified-response. I suspect that similar problems might also be elsewhere.
I am not sure what a better solution would be. My idea is that it would be better to have standard lowercase headers everywhere and prettify response headers with Camel-Case in ring adapter.
https://github.com/ring-clojure/ring/blob/master/ring-devel/src/ring/middleware/reload.clj
It appears wrap-reload needs to be passed a handler as normal, but with the caveat that its referenced through a var (see mmcgrana#72).
Something along the lines of:
(def app
(-> (compojure.core/routes my-routes)
(compojure.handler/api)))
(def app-with-reload
(ring.middleware.reload/wrap-reload #'app))
(defn -main []
(ring.adapter.jetty/run-jetty #'app-with-reload {:port 8081 :join? false})
(ring.adapter.jetty/run-jetty #'app-with-reload {:port 8082 :join? false}))
If instead wrap-reload is treated as a normal middleware—placed in the chain for app—it seems to work but the updated behavior is actually delayed by one request.
Raising this issue to hopefully get some documentation in wrap-reload that makes note that it needs the var reference; also it may be helpful for it to throw an exception if it's not passed a var to prevent accidental use as typical middleware.
ring/ring-core/test/ring/util/test/response.clj
line 126: (let [resp (file-response "backlink/foo.html"
backlink is not directory.
wrap-resource
is broken when used from a JAR file created with Leiningen 2.3.3. When the root URL is requested, wrap-resource
thinks it can find an empty resource called "" and serves that instead of passing the request on to the handler.
I've created a SSCCE demonstrating the problem: https://github.com/noidi/wrap-resource-lein-2.3.3
I suspect the bug is caused by this change made in Leiningen 2.3.3: "Add directory entries to jar files."
I would recommend using the latest tools.reader contrib library at https://github.com/clojure/tools.reader, functions clojure.tools.reader.edn/read or read-string, since they only require Clojure 1.3, unlike Clojure 1.5's clojure.edn namespace. Check their APIs before just doing a simple replacement, since there are slight differences, but perhaps not for the one-arg calls you are using in ring.
The use in ring-core/src/ring/middleware/cookies.clj function normalize-quoted-strs might be safe already because it is only done on strings that begin with a double-quote, but it would still be better to replace that call, too.
Other occurrences that should be replaced:
File ring-core/src/ring/middleware/session/cookie.clj:
function serialize postcondition
function unseal
Motivation: With Clojure 1.3 and 1.4, even with read-eval bound to false, clojure.core/read and read-string can cause arbitrary Java constructors to be executed, a few of which have potentially dangerous side effects that an attacker could use:
;; This causes precious-file.txt to be created if it doesn't exist, or
;; if it does exist, its contents will be erased (given appropriate
;; JVM sandboxing permissions, and underlying OS file permissions).
(defn read-string-unsafely [s](binding [read-eval false]
%28read-string s%29))
(read-string-unsafely "#java.io.FileWriter["precious-file.txt"]")
Typically you only want wrap-stacktrace
to take effect in a development setting. Doing this with environ looks something like this:
(jetty/run-jetty (-> #'app
((if (env/env :dev)
trace/wrap-stacktrace
identity))
[...])
{:port port :join? false})
If wrap-stacktrace
took a second argument to turn it into a no-op, it would be easier to use in the context of ->
.
(jetty/run-jetty (-> #'app
(trace/wrap-stacktrace (env/env :production))
[...])
{:port port :join? false})
What do you think?
In SPEC, the :content-type
key is defined to be the mime-type of the request.
Most ring adapters (like jetty, http-kit and http-core) just copy the contents of the Content-Type header verbatim.
The netty ring adapter however lowercases this and strips charset and multipart-boundary information.
It would seem to me netty-ring-adapter does the right thing according to SPEC: it computes just the mime type of the request.
However: multipart-params middleware requires the boundary, but doesn't (get-in request [:headers "content-type"])
to get the raw header.
:content-type
in the request map if it's just a direct copy out of :headers
? Convenience? Then you still have to remove ";" etc if you want to get the mime-type.In multipart_params there is a reference to assoc-conj in ring.util.codec. As far as I can tell there is no such namespace anymore. Maybe the function has moved to another namespace?
So, I have a toy clojure program using Compojure routes and Ring Middleware ([ring/ring-core "1.2.0"]
). The wrap-keyword-params
doesn't seem to like arrays very much.
I have some client-side javascript which calls:
$.get('/test', {"array": [1,2,3], "egg": "spam"});
The toy clojure code:
(ns toy
(:require [ring.middleware.keyword-params :refer :all]
[compojure.core :refer :all]))
(defn wrap-routes [all-routes]
(-> all-routes
wrap-keyword-params))
(defn test-route [{params :params}]
(println params)
{:status 200})
(defroutes all-my-routes
(GET "/test" r (test-route r))
;; snip
)
(def all (wrap-routes all-my-routes))
;; snip
And the output is precisely {array[] [1 2 3], :egg "spam"}
.
I suspect this is because in ring.middleware.keyword-params
the test (keyword-syntax? k)
on line 12 is incorrect for arrays. The keyword may resemble "array[]", as in my minimal code example, which then fails the test for keyword syntax.
Is this a bug, or am I doing something horribly wrong?
Addendum. The "obvious" solution, wrappying my routes with:
(defn wrap-routes [all-routes]
(-> all-routes
wrap-nested-params
wrap-keyword-params))
Well, it produces {array [1 2 3], :egg "spam"}
, which is a step in the right direction, but still off.
Documentation for ring.middleware.cookies (https://github.com/ring-clojure/ring/wiki/Cookies) states that by using :port attribute handler may restrict cookies to a specific port, nevertheless browsers (tested with Chrome 26 and Firefox 24) pass cookies to servers listening on different ports.
ring-core/src/ring/middleware/cookies.clj uses `Port=' attribute from RFC2965 obsoleted by RFC6265 (https://tools.ietf.org/html/rfc6265) which manifest:
Cookies do not provide isolation by port. If a cookie is readable by a service running on one port, the cookie is also readable by a service running on another port of the same server.
(ns ring-session-test.core)
(use 'ring.middleware.session)
(defn handler [{session :session uri :uri}]
(let [res {:status 200
:headers {"Content-Type" "text/plain"}
:body (str session)
:session {:a 1}}]
(if (= (session :a) 1)
(assoc res :session-cookie-attrs {:max-age 3600})
res)))
(def app
(wrap-session handler))
What I wanted is that the session cookie is sent on the first request
and then set the expiry date on another one. But on the second
request there is no cookie sent - I copy the headers below. If I add
:session-cookie-attrs on the first response - then it gets there in
the Cookie:
Set-Cookie:ring-session=00bd9357-a388-4714-ac1b-4e66def7e2f7;Max-Age=3600;Path=/
But on the second not.
Request URL:http://localhost:3000/che
Request Method:GET
ring-jetty-adapter
still uses Jetty 7 which has been superseded by Jetty 8 & 9. Is it desirable to upgrade ring-jetty-adapter
to Jetty 9 and Servlet API 3.0? If so, I will be more than happy to create a patch for this.
(def -main
(run-jetty #'app {:port 3000 :join? false}))
and this is the line that gives me the warning, but server still works.
The Ring specification states that the response body can be an ISeq of strings. Wouldn't it make sense to let it be an ISeq of {String, ISeq, File, InputStream} instead? This would make composition of response bodies much simpler; a middleware that wishes to decorate a response body can then simply do (vector "callback(" body ");")
without caring about whether body is a string or not.
Stack trace:
java.lang.NullPointerException
at ring.middleware.multipart_params$request_context$reify__1899.getContentLength(multipart_params.clj:23)
at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.(FileUploadBase.java:936)
at org.apache.commons.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:331)
at ring.middleware.multipart_params$file_item_seq.invoke(multipart_params.clj:38)
at ring.middleware.multipart_params$parse_multipart_params.invoke(multipart_params.clj:54)
at ring.middleware.multipart_params$wrap_multipart_params$fn__1919.invoke(multipart_params.clj:98)
In Windows 7 + JVM(1.6.0_35) environment, download site using ring-server has been set-up.
archive.zip exceeding 100Mbyte has been used.
If archive.zip is downloaded using browser or wget then file transfer stops halfway.
But, it works fine in ubuntu environment.
project.clj
(defproject sample4 "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.4.0"]
[org.clojure/tools.cli "0.2.2"]
[ring/ring-core "1.1.6"]
[ring/ring-jetty-adapter "1.1.6"]
[compojure "1.1.3"]]
:plugins [[lein-ring "0.7.5"]]
:ring {:handler sample4.route/all-routes}
:profiles {:dev {:dependencies [[ring-mock "0.1.3"]]}})
route.clj
(defroutes app-routes
(GET "/" [] "Hello World")
(GET "/archive.zip" []
(let [archive-file (java.io.File. "c:\\home\\sample\\archive.zip")]
{:status 200
:body archive-file
:headers {"Content-Type" "application/zip"
"Content-Length" (str (.length archive-file))
"Cache-Control" "no-cache"
"Content-Disposition" (str "attachment; filename=" (.getName archive-file))}}
)
)
(route/not-found "Not Found"))
What bug is this?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.