Algo fetch: HTTP errors aren't displayed in the logs about substra-backend HOT 4 CLOSED

It looks like it is similar to the issue #217. The cause of the failure is different, but it's the same ambiguous error message.

from substra-backend.

After more investigation, the case I was looking at was not a HTTP 403 error. Despite the appearances, the server returns a 200.

For context, this is what the worker pod receives when it tries to fetch e.g. susbtra-backend.node-1.com/algo/.... but its DNS resolution points to the public internet, and not the k8s cluster.

Maybe another approach would be to look at the response content type. Thoughts @Kelvin-M

from substra-backend.

I'm going to close this for now as there's no easy way to fix it.

We should keep in mind that errors like hash doesn't match ... are usually due to DNS misconfiguration. The solution is make sure that the address substra-backend.node-{1-N}.com resolves to the cluster IP, and not to the public internet. This could be part of a troubleshooting guide? cc @natct10

from substra-backend.

I'm going to close this for now as there's no easy way to fix it.

We should keep in mind that errors like hash doesn't match ... are usually due to DNS misconfiguration. The solution is make sure that the address substra-backend.node-{1-N}.com resolves to the cluster IP, and not to the public internet. This could be part of a troubleshooting guide? cc @natct10

You're right! I will add this to the troubleshooting section, thank you for pointing this one out @AurelienGasser

from substra-backend.