racket / web-server Goto Github PK

The WebSocket implementation of this library is outdated and no longer works. Fortunately, a new implementation created by @tonyg already exists at tonyg/racket-rfc6455. Can we merge that into this package?

Recently, people have pointed out that the combination of common caching proxies and web frameworks that treat ; as a query separator can lead to security problems; see here: https://snyk.io/blog/cache-poisoning-in-popular-open-source-packages/

This led Python to change the default behavior: https://bugs.python.org/issue42967

See also this article: https://lwn.net/Articles/846847/

We have a few choices here:

1. Do nothing (probably a bad idea)
2. Enable users of the web server to disable ; as a separator.
3. Like 2, but make that the default. (This is what Python did)

and is really useful for selecting.

The documentation on using Apache with the Racket Web Server currently recommends mod_rewrite:

However, the extended documentation for mod_rewrite lists Simple Proxying as an example of when not to use mod_rewrite: it recommends ProxyPass from mod_proxy.

I was reminded of this while writing #126 (comment). I'm not sure if I have a working example to contribute at the moment (I'm in the midst of some configuration changes locally), but I'll send one if I do.

When I try to print or display something in a servlet, I don't see it in my terminal.

(serve/servlet (λ (req) (print req) (response/xexpr `(html))))

Why is this? And if it's intended, it should be documented.

ACTUAL: The program below asks for SSL service, and starts successfully, but when visited via https://localhost:5555/, it complains about malformed requests. When visited via http://localhost:5555/, it runs the servlet and serves HTML successfully.

EXPECTED: http service should fail; https service should work, and should be properly TLS-encrypted.

REPRODUCTION:

First, generate keys:

        openssl genrsa -des3 -passout pass:a -out private-key.pem 1024
        openssl rsa -passin pass:a -in private-key.pem -out private-key.pem
        openssl req -new -x509 -nodes -sha1 -days 365 \
                -subj /CN=demo.server.example \
                -passin pass:a \
                -key private-key.pem > server-cert.pem

Then, run the following Racket program:

#lang racket/base

(require web-server/servlet)
(require web-server/servlet-env)

(serve/servlet (lambda (req) (response/xexpr `(html (body (h1 "Hello")))))
               #:launch-browser? #f
               #:quit? #f
               #:listen-ip #f
               #:port 5555
               #:ssl? #t
               #:ssl-cert (build-path (current-directory) "server-cert.pem")
               #:ssl-key (build-path (current-directory) "private-key.pem")
               #:servlet-regexp #rx"")

What version of Racket are you using?
8.3 [cs]

What program did you run?
I have a programme include webserver.rkt, b.rkt and c.rkt. And a.rkt require b.rkt and c.rkt and other lib like web-server provided by racket. I tried to build a excutable version through raco but failed to run it.
raco exe a.rkt
I run it,
./a and
path->collects-relative: contract violation expected: path-string? given: ''#%embedded:web-server/formlets/lib: context...: body of '#%embedded:web-server/formlets/lib:

What should have happened?
It should run correctly without error like when I run a source

If you got an error message, please include it here.
path->collects-relative: contract violation
expected: path-string?
given: ''#%embedded:web-server/formlets/lib:
context...:
body of '#%embedded:web-server/formlets/lib:

Please include any other relevant details
Operating system: debian and windows

I found a single
(require web-server/formlets)
would cause a error after compiling and running

This is going to be a long story. It leads to a bad bug, which took me probably more than 5 years to debug since I had to do that indirectly through enthusiastic students that are hard to find.

The end result of the below is that download urls from the handin server which go through Apache which serves HTTP2 just don't work at all in Safari, which will retry the download many times and eventually give up. The (easy + verified) fix is as the title says, to avoid duplicating the header if it's already included in the user call. Pointers below.

More details:

The thing that happens --- as much as I can guess --- is that the handin server creates a download URL with a Content-Length header in this bit of the code, and the web server blindly adds a second Content-Length in this code through this function which doesn't check if it exists in headers before adding it (and therefore the fix would be in this place).

The nasty aspect of the resulting brokenness has several aspects that makes it insane to figure it out:

• I'm 75% sure that it started happening when I switched my apache config to do HTTP2. The handin server is piped out through apache. This is on an ancient Fedora installation, and an awkward https setup --- so these could lead to the problem too, but looking at the above code, I think that the racket web server will actually duplicate the header.

• The brokenness only happens using Safari on macos. In their infinite wisdom, a certain OS maker decided to drop Safari on any platform other than their own, which means that I can't even debug the problem without owning one of the devil machines that some people are very fond of (I'm still holding to the same opinion that I'd rather hit the nearest wall with my head before I let one of these damned machines into my home).
  (This is a 100% certainty, it doesn't happen on other OSs/browsers.)

• Trying to look at the client-server chatter through devtools doesn't show the problem since it keeps one copy of the header.

• Trying to use curl also fails: having the duplicate header and running it with -i leads to:

  curl: (92) HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)
  

Since this is so obscure, and requires a macos to see the effect, I just told students to use chrome instead. (And given my vocal opinions about apple, I had to explain that this is not intentional --- I'm not going to touch it, but I didn't intend to stand in-between some students' pistols and their respective feet.) Every couple of semesters or so, some student would try to debug it, fail to see the problem, get on a zoom with me so I can have the pleasure of looking at HTTP headers while scratching my own head. That went on for a number of years.

Finally, I had a student that happily jumped into a wireshark experiment, and finally caught the problem.

I verified that removing the header creation in the handin code works -- at least as much as using curl shows (no error when it doesn't create the header).

CC @mflatt -- removing that bit from web-status-server.rkt will resolve it now, but fixing the web server would be nice too.

The web-server/http/cookie and web-server/http/id-cookie libraries (and, to a lesser extent, web-server/http/cookie-parse) are implemented using net/cookie, which carries a big scary warning about being deprecated in favor of net/cookies from the net-cookies package, which implements RFC 6265.

I'm not sure what the web server should do about this. For my own purposes, I've more or less implemented the API of the web-server cookie libraries using net/cookies. I'm happy to clean up and share that code, but I'm not sure if it should be a package or a contribution to the web server and, if the later, what the backwards-compatibility considerations are.

If the conclusion is that the web server should continue to provide the current libraries based on the deprecated net/cookie, it seems like the web server documentation should also note the limitations of net/cookie in appropriate places. It seems particularly relevant to web-server/http/id-cookie, as users probably want features like HttpOnly.

When the web server accepts a SSL connection, the web server (in its main loop):

• accepts the connection
• negotiates the SSL tunnel, without any timeout on this process.
• then creates a new thread to handle the connection.

This has the problem that a malicious client could just effectively hang the web server by opening a connection to it while not negotiating the SSL connection.

Steps to reproduce:
(in one terminal) plt-web-server --ssl -port 5000
(in another terminal) run telnet localhost 5000, do nothing.
(in another terminal) openssl s_client --connect localhost:5000 (this will hang until you close telnet)

The rendering of signatures in the Continue tutorial is confusing to readers

In the tutorial, functions are often described, then followed by their signature in the familiar scribble/manual format.

Unfortunately the hairline at the top of the signature, in addition to the large gap above, and lack of a gap below gives the impression the signature is associated with the subsequent text.

e.g. http://docs.racket-lang.org/continue/index.html?q=specform#%28def._%28%28lib._web-server%2Fservlet..rkt%29._extract-binding%2Fsingle%29%29

My suggestion is the rendering of continue is changed to support the 'description followed by signature' style of the text. I'd suggest moving the hairline to the bottom of the signature, and increasing the spacing between the signature and the subsequent paragraph.

Options to resolve;

• a different css stylesheet for continue.scribble
• an new signature function that handles spacing differently
• other options?

https://github.com/racket/web-server/blob/master/web-server-doc/web-server/scribblings/tutorial/continue.scrbl

https://github.com/racket/scribble/blob/master/scribble-lib/scribble/private/manual-proc.rkt

While looking at a Discourse thread, I noticed that make-dir-store uses (with-output-to-file #:exists 'replace ...):

It seems like it would be better to use call-with-atomic-output-file here: if somehow you ended up with a partially-written file, it seems like it would be very painful to debug.

I recently discovered that dispatch-rules and related forms make it hard to handle trailing /s on URLs sensibly.

The URL https://example.com/foo/ (with a trailing /) is treated differently than https://example.com/foo (without a trailing /). Strictly speaking, this is correct: those are two distinct URLs, and in principle one could serve completely unrelated content at each. In practice, though, users don't expect such URLs to be distinct. Serving different non-error content at one than the other is a terrible idea, and even returning a successful response from one variant but, e.g., a 404 error from the other (as my sight was doing) is likely to cause confusion. I expect most programs will want to either redirect the less-preferred variant to the canonical variant or simply treat both variants as equivalent.

Unfortunately, web-server/dispatch doesn't provide a great way to implement such behavior. Here's an example in code of the current state of affairs:

#lang racket

(require web-server/dispatch
         web-server/http
         web-server/servlet-env
         net/url
         rackunit)

(define ((handler str) req)
  (response/output
   (λ (out) (write-string str out))))

(define start
  (dispatch-case
   [() (handler "a")]
   [("") (handler "b")]
   [("foo") (handler "c")]
   [("foo" "") (handler "d")]
   [("foo" "" "") (handler "e")]
   [("foo" "" "bar") (handler "f")]))

(define (do-request str)
  (port->string (get-pure-port (string->url str))))

(check-equal?
 (let ([th (thread
            (λ ()
              (serve/servlet start
                             #:servlet-regexp #rx""
                             #:banner? #f
                             #:launch-browser? #f)))])
   (sleep 5)
   (begin0 (list (do-request "http://localhost:8000")
                 (do-request "http://localhost:8000/")
                 (do-request "http://localhost:8000/foo")
                 (do-request "http://localhost:8000/foo/")
                 (do-request "http://localhost:8000/foo//")
                 (do-request "http://localhost:8000/foo//bar"))
           (kill-thread th)))
 '("b" "b" "c" "d" "e" "f"))

This illustrates a few things:

• Because of how HTTP requests work, the root URL is always equivalent with or without a trailing / and has a single, empty path element ("").
• () is allowed as a pattern, but (due to the above) will never match anything. I think it might be better to make this a syntax error (or a warning that will become an error in a future release).
• A trailing / adds an empty path element ("") to the end of the URL.
• There is no cleanse-path-like case for multiple adjacent / separators.

I have changed my application to handle trailing / separators by using dispatch-rules+applies instead of dispatch-rules (which involved removing my else clause) and, if the original request does not satisfy the predicate from dispatch-rules+applies but a version with a normalized path would, responding with a redirect to the normalized path.

I think it would be better to extend web-server/dispatch to support this sort of thing, but I'm not sure yet what the best way would be to do so. A few things I've been thinking about so far:

• The current language for dispatch-rules patterns doesn't have a notion of "splicing" patterns: each string literal or bidi-match-expander use applies to a single path element.
• Simply giving the same response to all variants seems relatively straight-forward to add as a keyword option. However, that is (at least arguably) less optimal than redirecting to the canonical URL. Redirects open another can of worms, though, since 301 Moved Permanently has issues for methods other than GET and HEAD and 308 Permanent Redirect doesn't have a straightforward fallback for older browsers. I would want to do a 301 Moved Permanently for GET and HEAD requests and 307 Temporary Redirect for other methods, which RFC 7231 suggests is reasonable. The higher-level question is the trade-off between a simple API that "does the right thing" and configurability.
• Building on that theme, there are some cases other than trailing / support where a similar ability to handle variants on canonical URLs would be nice: for example, case-insensitivity. I can imagine possible extensions to the dispatch-rules API to add a general concept of non-canonical URLs, and it is appealing from one perspective to avoid making trailing / support a baked-in special feature. On the other hand, though, there seems a risk of getting beyond the scope of this library.

The docs suggest using 1024-bit keys:

openssl genrsa -des3 -out private-key.pem 1024

But these days that produces an error as being too short. Even 2048 doesn't seem long enough. Take it up 4096.

When the initial-connection-timeout expires, the web server closes some ports and shuts down a custodian. However, this can take a surprisingly long time. In particular, the calls to close-{input,output}-port can add 500+ms of latency! It's not clear to me why this happens, but maybe there's something else we can do here.

To reproduce, see https://github.com/zkry/racket-test-server and note the spikes in the resulting picture. They correspond exactly to this timer, and if you change the initial timeout to 2, they happen every 2 seconds. If you comment out the two with-handlers forms in kill-connection-w/o-timer!, the spikes go away.

http://drdr.racket-lang.org/54239/cs/racket/share/pkgs/web-server-test/tests/web-server/run-all-tests.rkt

This particular test has been failing occasionally, but seemingly only on Racket CS.

--------------------
Connection Manager > Internal > Web Server > Extend timer
FAILURE
name:       check-true
location:   private/connection-manager-test.rkt:51:4
params:     '(#f)
--------------------

The contract for response/output is incorrect: the #:message keyword argument should be (or/c bytes? #f) rather than just bytes? as it currently is.

Should call-with-web-prompt be a documented export of web-server/lang/abort-resume?

It is mentioned here, and I have used it at least once — but I'm asking rather than documenting as I'm not sure if it's intended to be a "documented" part of the API, with everything that entails.

I think that Continue, as a flagship tutorial for Racket, needs new sections to cover changes in web development practice especially supporting client side web frameworks(angular,Ember,Vue,React) by building backends that support XMLHttpRequest/WebSockets/Fetch. Other possibilities include

• developing with client side racket (RacketScript, Urlang, Whalesong?)
• authentication (both doing it yourself, or using OpenID to)
• creating and accessing web API's like AWS, twitter etc.

I think the best way to do this would be to develop a tutorial in the style of Continue, then it could be linked from, or integrated into the Continue tutorial.

Any thoughts? Is this a good or bad idea? Would it be better to focus community effort elsewhere?

Kind regards,

Stephen

After refactoring a big web-server program, attempting to run it now blames one of my modules for violating the contract of with-current-saved-continuation-marks-and ("expected 1 value, returned 2 values in: the range of the 3rd argument of (-> any/c any/c (-> any/c) any/c)"—full message below).

I am not sure how to minimize this example. The blamed module doesn't use with-current-saved-continuation-marks-and directly (nor does any of my code), and it doesn't even seem to use or define any functions with multiple return values. It is written in a language based on #lang web-server. (Changing to the plain #lang web-server doesn't seem to help.)

The bug seems to be the web server's fault: changing the blamed module to use #lang racket/base seems to fix the error. (It isn't clear why this module was in a web-server-based language in the first place: my guess is that I left it that way in an earlier refactoring.)

The contract problem happens both in Racket 7.4 CS and in traditional Racket built from master, but Racket CS has an additional problem: if I try to run the blamed module directly (i.e. racket portal/private-servlet/feedback/notify.rkt) when it's #lang is either web-server or my web-server-based language, Racket 7.4 CS has an "invalid memory reference," whereas traditional Racket succeeds.

Here's the full error message:

Sapientia:ricoeur-portal philip$ ./config/no-voyant-test.rkt 
with-current-saved-continuation-marks-and: contract violation;
 expected 1 value, returned 2 values
  in: the range of
      the 3rd argument of
      (-> any/c any/c (-> any/c) any/c)
  contract from: 
      <pkgs>/web-server-lib/web-server/lang/abort-resume.rkt
  blaming: <pkgs>/ricoeur-portal/portal/private-servlet/feedback/notify.rkt
   (assuming the contract is correct)
  at: <pkgs>/web-server-lib/web-server/lang/abort-resume.rkt:244.2
  context...:
   /Users/pltbuild/build/plt-release-64/bundle/racket/collects/racket/contract/private/blame.rkt:347:0: raise-blame-error16
   loop
   /Users/pltbuild/build/plt-release-64/bundle/racket/collects/racket/contract/private/arrow-common.rkt:63:5
   call-with-values
   call-with-values
   call-with-values
   /Users/philip/code/ricoeur/ricoeur-portal/config.rkt:207:22: pr:proc
   call-with-values
   /Users/pltbuild/build/plt-release-64/bundle/racket/collects/racket/private/promise.rkt:103:0: force/generic
   /Users/philip/code/ricoeur/ricoeur-portal/config.rkt:217:21
   call-with-values
   body of (submod "/Users/philip/code/ricoeur/ricoeur-portal/config/no-voyant-test.rkt" main)
   temp37_0
   for-loop
   run-module-instance!125
   #%for-each

The htmxl function in web-server-lib/web-server/test.rkt splits a response into headers and body parts with regexp ^(.+)\r\n\r\n(.*)$. If the body part contains '\r\n\r\n', the first matching group takes a piece of body as + is greedy quantifier. Please replace the regexp with ^(.+?)\r\n\r\n(.*)$

It looks like read-headers disappeared from the provides of web-server/http/request sometime since the branch for v7.5. That breaks the "rfc6455" package, at least.

A few places in the web server use cryptographic hash functions, specifically MD5 and SHA1:

• md5-stuffer
• HMAC-SHA1 and HMAC-SHA1-stuffer from web-server/stuffers/hmac-sha1
• All of the functions from web-server/http/id-cookie are built on HMAC-SHA1.

Neither MD5 nor SHA1 are recommended anymore for general use as cryptographic hash functions. IIUC, the vulnerabilities in both cases are (so far) only with collisions, not preimages, which I think means some or all of these uses are still ok—but "I think" is not something I like to rely on when it comes to crypto.

I propose that:

1. We should document the security considerations applicable to each use of cryptographic hash functions.
2. If MD5 or SHA1 are insecure in any of these applications, we should replace them with better hash functions. Conveniently, racket/base now provides sha256-bytes and sha254-bytes.

When servlet instances are flushed (selectively or not), the only action taken is that servlet instance references are discarded. In particular, resources managed by the servlet custodian are not shut down but are instead managed (directly) by the superordinate custodian.

Should the servlet custodian be shut down when the instance is flushed?

For my use case (spawning a service thread), it looks like I can get by with a will, but shutting down the custodian seems like a more “natural” general policy. (And maybe it is, but is also a breaking change. Is that the case?)

Sometime between Racket 6.0 and 6.3 plt-web-server --ssl stops creating a server that accepts SSL connections. On accepting an SSL connection, read-request errors out with malformed request:

read-request: malformed request ��W+���)e�┼!��ʟԾ^��)4�3�E��� ʷˤ!�H~��q����7ɍ#)<ȁ����]Fr�J��$�#�
context...:
/usr/racket/share/pkgs/web-server-lib/web-server/private/util.rkt:37:0: network-error
...ow-val-first.rkt:302:18
/usr/racket/share/pkgs/web-server-lib/web-server/http/request.rkt:31:0
/usr/racket/share/pkgs/web-server-lib/web-server/private/dispatch-server-with-connect-unit.rkt:131:8

How to add by (response/xexpr) ?
I cannot do it by (response/xexpr '(!DOCTYPE html (html ...))) or (response/xexpr '(!DOCTYPE ((html)) (html ...)))

The instance-expiration-handler of the default manager constructed by dispatch/servlet will returns an xexpr? instead of a response?.

Exception

The application raised an exception with the message:

make-threshold-LRU-manager: contract violation
  expected: can-be-response?
  given: '(html (head (title "Page Has Expired.")) (body (p "Sorry, this page has expired. Please go back.")))
  in: the range of
      a part of the or/c of
      the 1st argument of
      (->
       (or/c #f (-> request? can-be-response?))
       number?
       manager?)
  contract from: 
      <pkgs>/web-server-lib/web-server/managers/lru.rkt
  blaming: <pkgs>/web-server-lib/web-server/servlet-dispatch.rkt
   (assuming the contract is correct)
  at: <pkgs>/web-server-lib/web-server/managers/lru.rkt:16.2

Stack trace:

raise-blame-error16 at:
  line 347, column 0, in file /Users/ayman/Applications/Racket-v7.4/collects/racket/contract/private/blame.rkt
<unknown procedure> at:
  line 364, column 33, in file /Users/ayman/Applications/Racket-v7.4/collects/racket/contract/private/arrow-higher-order.rkt
select-handler/no-breaks at:
  line 163, column 2, in file /Users/ayman/Applications/Racket-v7.4/collects/racket/private/more-scheme.rkt
<unknown procedure> at:
  line 360, column 33, in file /Users/ayman/Applications/Racket-v7.4/collects/racket/contract/private/arrow-higher-order.rkt
<unknown procedure> at:
  line 63, column 2, in file /Users/ayman/m/packages/web-server/web-server-lib/web-server/dispatchers/dispatch-servlets.rkt
select-handler/no-breaks at:
  line 163, column 2, in file /Users/ayman/Applications/Racket-v7.4/collects/racket/private/more-scheme.rkt
<unknown procedure> at:
  line 144, column 8, in file /Users/ayman/m/packages/web-server/web-server-lib/web-server/private/dispatch-server-with-connect-unit.rkt

The implementation of formlet/c, in particular the use of quote-this-module-path, leaves that absolute module path in the resulting zo file. I think this can be fixed by using

(define quote-this-module-path (path->collects-relative-path (quote-module-path)))

but I'm not sure exactly what the intended goal is here, so I'm not sure.

when I enter a number, it went through the second url but it still show up "Enter the First number to add:". Then when I type another number and hit submit, it won't go to display-sum.

When using the unix-domain-socket tcp unit, broken pipe (errno 32) exceptions are not being caught and handled because they have type exn:fail:filesystem:errno , and the dispatcher is only handling exn:fail:network:errno

This is only an annoyance (exception spam in the console)

When making a request with a path like this:

GET //foo/bar HTTP/1.1

…the Racket web server will naïvely call string->url on the path, producing a URL with foo as the host and bar as the only path segment. This is incorrect, since the correct parsing would be a URL with no host and three path segments, the first of which is the empty string.

@soegaard reported on Slack that the contract on make-id-cookie currently requires the name and value arguments to be given as strings, so this program fails:

#lang racket

(require web-server/http/id-cookie
         racket/random)

(make-id-cookie #"aname" #"avalue"
                #:key (crypto-random-bytes 128))

All other cookie-constructing functions permissively accept byte-strings for names and values (converting them with bytes->string/utf-8), and it seems like make-id-cookie should, too—I probably just missed it after I figured out RenaissanceBug/racket-cookies@c745ee5. I will try to make a PR soon.

The SSL Labs "SSL Server Test" service (https://www.ssllabs.com/ssltest/) identifies some aspects of the Racket web server's default HTTPS configuration that should be improved. Most significantly, it says, "This server does not support Forward Secrecy with the reference browsers. Grade capped to B."

I am still looking into the situation in more detail, but I've noticed at least two differences from the configuration generated by Certbot for Apache, which SSL Labs approves of:

1. While the Racket web server supports ECDHE, it doesn't seem to prefer more secure cypher suites to less secure ones.
2. The Racket web server's default configuration doesn't seem to enable DHE. It seems like this would force clients that support DHE but not ECDHE to fall back to RSA key exchange without forward secrecy.

I'm happy to do some implementation work here, but I haven't worked with these low-level portions before. In particular, I haven't figured out how to designate preferred cypher suites with the Racket openssl module.

The value of (current-directory) upon entry to start is the htdocs directory inside the web-server collection, not the directory that Racket started in (as for other Racket programs) or the directory containing the program.

Here's a list of improvements that I wish to see:

Better spacing

The following image shows one of the issues.

There's too much space before the box, and not enough space after the box.

Consistent usage of contract

There's a weird mix of HtDP-style signature and actual Racket contract throughout the tutorial, which is confusing. We should pick one style and use it consistently.

Slightly off-topic, but while I know that the tutorial's target audience is readers of the first edition of HtDP, I'm not sure if that makes sense anymore.

Use appropriate tags

Several @tt should have been @litchar.

Linkify code

All links under #lang web-server/insta are broken.

Consistent naming

Right now, functions whose name is prefixed with render- are used in two different ways: they might return an xexp/c or a response?. It would be nice to make things consistent. Perhaps make-XXX-request for functions that return response? and render-XXX for functions that return xexpr/c?

Avoid "request" as an argument name.

This is to avoid incorrect linkified text.

From RFC2616 Hypertext Transfer Protocol -- HTTP/1.1, section 4.2 (link):

Each header field consists of a name followed by a colon (":") and the field value. Field names are case-insensitive.

It appears there is a headers-assq which is case-sensitive and a headers-assq* which is not case-sensitive.

Would it make sense to merge these two procedures into one that is strictly case-insensitive?

I was walking through the continue tutorial and I think on macos, with the new strictness about allowing things to write to the file system, having to grant special permissions to specific apps, I’ve run into a problem with the tutorial. Here is a screenshot. I'm trying to figure out what would need to be granted write permission and how/where to do that.

Because

Example:

(define (start req)
  (parameterize ([current-unescaped-tags html-unescaped-tags])
    (response/xexpr
     `(html (head (script "let x = 1 < 2;"))))))

When run, the above will fail because it will convert the < to < when that isn't the intention. I'd recommend changing the output function to first capture - and then reuse - the value:

(let ([unescaped-tags (current-unescaped-tags)])
  (λ (out)
    (parameterize ([current-unescaped-tags unescaped-tags])
      (write-bytes preamble out)
      (write-xexpr xexpr out))))

But there may be something a bit lower level that could work as well.

If you make a dispatcher with dispatch-files and submit non-GET requests to it, you'll find that you don't get a response at all. The connection stays open, but the server doesn't send anything over the wire after emitting the headers. Consider this server, which is intended to serve up whatever is in the current working directory in which it is run:

#lang racket/base

; Dispatcher created with dispatch-files hangs (does not output response)
; on POST requests

(require
 web-server/http/xexpr
 web-server/web-server
 web-server/dispatchers/filesystem-map
 web-server/servlet-dispatch
 (prefix-in s: web-server/dispatchers/dispatch-sequencer)
 (prefix-in f: web-server/dispatchers/dispatch-files))

(define (fallback req)
  (response/xexpr '(html (body (p "Didn't work.")))))

(define dispatcher
  (s:make
   (f:make #:url->path (make-url->path (current-directory)))
   (dispatch/servlet fallback)))

(define stop-serving-cwd
  (serve #:dispatch dispatcher
         #:port 8080))

(with-handlers ([exn:break? (lambda (err) (stop-serving-cwd))])
  (sync/enable-break never-evt))

Actual: If you put a file, say, foo.txt in the directory in which you run this program, and do a GET request for it, you'll see the file. If you, say, POST to /foo.txt, you won't get any response at all; the server just hangs.

Expected: fallback gets called, and I see a "Didn't work." response.

The source of the behavior is in output-file, defined in web-server/http/response.rkt. There, in the definition of output-file/boundary, which is called by output-file, I see this comment:

; TODO: What if we want to output-file during a POST?

I propose tackling this TODO.

My suggestion is to keep the definition of output-file in response.rkt unchanged and focus on the static file dispatcher. I propose changing dispatch-files.rkt so that it accepts only three HTTP methods: GET, HEAD, and OPTIONS. Invoking the files dispatcher with any other method would trigger a call to (next-dispatcher).

The proposed change would be largely backwards compatible. Method signatures wouldn't change. GET requests would continue to work as before. If one relied on HEAD and OPTIONS requests for static files not working, then this change would break that.

I'm happy to do the work here. I think the solution is straightforward to implement, but this page can serve as a place for discussing the issue.

Also worth thinking about:

• One could also consider dropping the HTTP method check in output-file/boundary. The function is essentially specialized on the method argument: it does something only when the method is #"GET" or some case-insensitive variation thereof.

The following program:

#lang web-server ;racket

(require web-server/formlets
         web-server/http
         net/url)

(define the-formlet
  (formlet "Something"
           (values 1 2 3)))

(define (foo)
  (formlet-process the-formlet
                   (request #"get"
                            (string->url "http://www.example.com")
                            null
                            (delay null)
                            #f
                            "192.168.1.1"
                            80
                            "192.168.1.2")))

(foo)

produces the error:

result arity mismatch;
 expected number of values not received
  expected: 1
  received: 3
  values...:
   1
   2
   3

If you change the language to #lang racket, the error goes away, so I don't think the formlet library is to blame, but I haven't found a simpler way to reproduce it. A simple example of multiple return values works fine, e.g.:

#lang web-server
(define (bar)
  (values 1 2 3))
(bar)

Avoiding formlet-process also seems to work:

> (let-values ([(xs p i) (the-formlet 0)])
    (p null))
1
2
3

The error occurs both with the stock 6.10 version of web-server-lib and with the latest version from Git.

The example in the documentation

(include-template #:command-char #\$ "dollar-static.html")

doesn't work in #lang racket/base out-of-the-box. One needs to also (require (for-syntax racket/base)). It might be helpful to include this in the documentation.

Just noting this down from workshop so I don't forget:

From https://docs.racket-lang.org/web-server/http.html?q=not-found#%28def._%28%28lib._web-server%2Fhttp%2Fresponse-structs..rkt%29._response%2Ffull%29%29

response/full docs did not clarify that the order of response headers provided is preserved. Came up when seeing this use that introduced some sugar for Set-Cookie: https://github.com/jessealama/racketcon-2018-web-devel-workshop/blob/master/respond.rkt#L193

Haven't used Scribble yet, but glad to take care of this one once I learn how to annotate the appropriate place.

Suppose I have a web application:

#lang racket
(require web-server/servlet-env)

; Some web app code here ...

;; Start the web server.
(serve/servlet request-handler
                       #:log-file "/dev/stdout")

When I start the web server and access the web app through the browser, the logs appear like this:

127.0.0.1 - - [25/Dec/2018:12:34:56 +0000] "GET /servlets/standalone.rkt HTTP/1.1" - -
127.0.0.1 - - [25/Dec/2018:12:34:56 +0000] "GET /favicon.ico HTTP/1.1" - -

The HTTP response code is missing from the logs, making it difficult to know when any of the web pages result in 500, 403, 404, etc.

Would it be possible to implement request + response logging? Request + response logs are more useful in tracking past errors. For example:

127.0.0.1 - - [25/Dec/2018:12:34:56 +0000] "GET /servlets/standalone.rkt HTTP/1.1"  401 152 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
127.0.0.1 - - [25/Dec/2018:12:34:56 +0000] "GET /favicon.ico HTTP/1.1" 404 152 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"

(Originally asked on Stack Overflow: https://stackoverflow.com/questions/53911162/)

I wanted to revisit the question raised in this old mailing-list thread about using #lang web-server/base for the formlets library so that formlets would be serializable.

The primary reason I personally would want this is because I often want to generate formlets dynamically, e.g. using something like checkbox-group to have a user select one of the results of a database query.

Is there a significant downside to doing this?

I tried executing this program using racket on macOS from iTerm:

#lang racket/base
(require web-server/servlet-env web-server/http)
(serve/servlet (λ (req) (print req) (response/xexpr `(html))))

It works in DrRacket, but not in the terminal. The output is printed only once the server is closed.

I just noticed the macro define-component in web-server/web-server-lib/web-server/lang/web-cell-component.rkt, which seems like it could be useful but doesn't seem to be documented anywhere (and it is less than immediately obvious how to use it).

Should it be documented?

It seems to be no easy way to protect static resources with web servlets nor specific routes, I made my way to protect routes with higher order functions but i can't find a way to prevent the static files from being served to any request, maybe there is a way that I am not aware of, maybe there is a way to handle the file serving part or inserting some sort of "middleware" to run before the file dispatch...

I would appreciate your help, thanks in advance.

I had a rough time getting stateless servers to work. The culprit appears to be web-server/servlet/web. Include it and stateless does not work, exclude it and it works quite seamlessly on a non-trivial system. See

https://discord.com/channels/571040468092321801/1128483815787077692

for a small, illustrative example, reproduced below.

However, I don't see anything in the documentation for either stateless or web-server/servlet/web that indicate this. It should be documented in both places.

Sample program:

#lang web-server

(require web-server/servlet-env
     web-server/servlet/web)

(define (start req)
  (println "starting")
  (let ([r (send/suspend
        (lambda (k-url)
          (response/xexpr
           `(html
         (body
          (form ([action ,k-url])
            (input ([type "text"] [id "uid"] [name "uid"]))))))))])
    (println "there")
    (let ([uid (extract-binding/single 'uid (request-bindings r))])
      (if (member uid '("hello"))
      (response/xexpr `(html (body (p "yes"))))
      (response/xexpr `(html (body (p "no"))))))))

(serve/servlet start
               #:listen-ip #f
               #:stateless? #t)