On `context deadline exceeded`, response headers aren't set about grpc-gateway HOT 3 OPEN

Hi, thanks for your issue! I'm a little confused about exactly what the bug is here. From what I can gather, this is the chain of events:

1. curl sends HTTP request to server
2. gRPC-Gateway handles the request
3. The request reaches the gRPC server
4. The store ID interceptor runs on the gRPC server, storing an ID in the ctx via grpc.SetHeader (not actually sure this works, since the use of the grpc-ecosystem interceptor pattern is making things hard to follow).
5. The gRPC server handler runs, which sleeps for 1 second, causing the gRPC client to time out the request.

Is this correct? Because as far as I'm concerned there is no way for the server to do anything while the handler is sleeping, and the gRPC client is timing out without waiting for a response. What do you think the gateway is doing wrong here? Nothing has been written on the wire for it to translate, and even if it was, I'm not sure it should.

Perhaps you could start by explaining what you were trying to do, and we can see if there's some other way to do it? I would recommend simplifying your test repo to just the essentials too (I assume the OTel observability stuff isn't necessary to exhibit the issue).

Thanks!

from grpc-gateway.

Hey @johanbrandhorst, thanks for the fast response 🙇‍♀️ I've trimmed the repo now 😄

I'm a little confused about exactly what the bug is here

It's likely that it's not a bug, more of a misunderstanding on my part on how to achieve my goal.

Perhaps you could start by explaining what you were trying to do

Yes, sorry, I wasn't clear. My goal is two-fold.

1. set a maximum amount of time that server handlers are allowed to run. (I thought runtime.DefaultContextTimeout = 50 ms achieved this)
2. send response headers back to clients in the case of errors (which includes the deadline being hit).

The store ID interceptor runs on the gRPC server, storing an ID in the ctx via grpc.SetHeader (not actually sure this works, since the use of the grpc-ecosystem interceptor pattern is making things hard to follow).

What part of the interceptor pattern is hard to follow? I thought this was the standard way of implementing middlewares? I can't think of an easier way of setting headers 🤔

In any case, my issue is that if you remove the time.Sleep in the handler (i.e. produce a success response) the response header is correctly received by the client. But if i artificially produce a deadline exceeded error, then the client doesn't get this header.

The gRPC server handler runs, which sleeps for 1 second, causing the gRPC client to time out the request.... Because as far as I'm concerned there is no way for the server to do anything while the handler is sleeping, and the gRPC client is timing out without waiting for a response

This is where i'm seeing cracks on my understanding. I thought that by doing

runtime.DefaultContextTimeout = 50 * time.Millisecond

i was setting a deadline on the server handler, not on the client. Is there a way of setting a deadline on the server?

from grpc-gateway.