Cannot connect to remote service - SERVICE_UNAVAILABLE about spring-cloud-kubernetes HOT 18 CLOSED

@wind57
Thank you this solves the issue. Previously this setting was not necessary when I used Eureka to discover services with Spring Gateway when I was not using Kubernetes.

from spring-cloud-kubernetes.

@wind57 Do you need additional information?

from spring-cloud-kubernetes.

1. there are no deployment files in your samples
2. even if I create RBAC files and deployment files and try to deploy it:

***************************
APPLICATION FAILED TO START
***************************

Description:

Config data resource 'file [/home/config/gateway-configuration.yml]' via location 'file:/home/config/gateway-configuration.yml' does not exist

Please create a sample that is reproducible. Thank you

from spring-cloud-kubernetes.

@wind57 I have updated the config files. I used helm charts. Please try to reuse your deployment files and RBAC files.

from spring-cloud-kubernetes.

First a disclaimer: I am not an employee of Spring team. I do this on my own time, without being payed. I hope you understand that. So every time I am asking for a reproducible/simple sample, is because that will save me a lot of time when investigating these kind of issues. That being said, your samples are by far not simple: I can't figure out the templating in helm (neither I should be doing that is a sample), so provide plain deployments/RBAC (I have been asking for this numerous times on your issues, all my requests where ignored); your application has far too many dependencies: vault, security, POST where a simple GET is needed, so on and so forth. Just strip down your application to the minimum.

I already explained what such a sample is on a different issue you opened, but here is a reminder: https://github.com/wind57/gateway-demo. This is what I am expecting, but not only me, any person willing to help you.

Here is what I had to do to re-produce the issue and fix:

• What I did is take the manifests from this sample : https://github.com/wind57/gateway-demo and adapt them to your project.
• In gateway project, I've added this in the application.yaml:

management:
  endpoints:
    web:
      exposure:
        include: gateway, health
  endpoint:
    gateway:
      enabled: true

This is needed so that I could go to /actuator/gateway and see what routes are there.

• I added : implementation 'org.springframework.cloud:spring-cloud-starter-bootstrap' in the gateway project also (this is most probably the core issue you were having)

• I deleted the WebClientConfig from gateway as it was not important to the issue

• I deleted the LoggingGatewayFilterFactory as it was not important the the issue

• I simplified Endpoint to being just:

@RestController
@RequestMapping(value = "/admin")
public class Endpoint {

    @GetMapping("/greeting")
    public @ResponseBody String greeting()
    {
        return "test response";
    }

}

• In the same project, I simplified application.yaml to:

spring:
  application:
    name: int-test-project
  jackson:
    time-zone: UTC

server:
  port: 8761

Then I deployed the apps, did a :

root@kind-control-plane:/# kubectl get pods -o wide
NAME                           READY   STATUS    RESTARTS   AGE     IP            NODE                 NOMINATED NODE   READINESS GATES
gateway-demo-775cb8956-rmmzk   1/1     Running   0          3s      10.244.0.14   kind-control-plane   <none>           <none>
mockup-786c547bcc-67cd2        1/1     Running   0          6m41s   10.244.0.12   kind-control-plane   <none>           <none>

and:

root@kind-control-plane:/# curl 10.244.0.14:8888/mockup/admin/greeting
test responseroot@kind-control-plane:/#

The most probably cause of your problem is that you are missing a dependency in gateway, read this issue (bottom comments)

from spring-cloud-kubernetes.

@wind57 Thanks. I removed much of the code that I have. Can you show me your rotes definition into gateway, please?

from spring-cloud-kubernetes.

In my sample, I don't have any explicit routes, all are created by the gateway based on the discovery client. You can see them by going to /actuator/gateway once you enable that endpoint (I showed above how to do that)

from spring-cloud-kubernetes.

@wind57 I tried this:

spring:
  application:
    name: gateway
  cloud:
    gateway:
      routes:
        - id: mockup
          uri: lb://mockup
          predicates:
            - Path=/mockup/**

But when I do GET <host>:8888/mockup/admin/greeting I get Not Found. Maybe I need to define it some other way?

I get these rotes:

[
    {
        "href": "/actuator/gateway/routedefinitions",
        "methods": [
            "GET"
        ]
    },
    {
        "href": "/actuator/gateway/globalfilters",
        "methods": [
            "GET"
        ]
    },
    {
        "href": "/actuator/gateway/routefilters",
        "methods": [
            "GET"
        ]
    },
    {
        "href": "/actuator/gateway/routes/mockup/combinedfilters",
        "methods": [
            "GET"
        ]
    },
    {
        "href": "/actuator/gateway/routes",
        "methods": [
            "POST",
            "GET"
        ]
    },
    {
        "href": "/actuator/gateway/routes/mockup",
        "methods": [
            "POST",
            "DELETE",
            "GET"
        ]
    },
    {
        "href": "/actuator/gateway/",
        "methods": [
            "GET"
        ]
    },
    {
        "href": "/actuator/gateway/routepredicates",
        "methods": [
            "GET"
        ]
    },
    {
        "href": "/actuator/gateway/refresh",
        "methods": [
            "POST"
        ]
    }
]

PS

/actuator/gateway/routes
[
    {
        "predicate": "Paths: [/mockup/**], match trailing slash: true",
        "route_id": "mockup",
        "filters": [],
        "uri": "lb://mockup",
        "order": 0
    }
]

from spring-cloud-kubernetes.

what if you delete your explicit routes?

from spring-cloud-kubernetes.

@wind57 Again not found with removed rotes.
Is it possible that services cannot communicate due to blocked access between pods?

from spring-cloud-kubernetes.

I dont understand what "blocked access" is, to be honest. Can you access the mockup pod by IP on its /admin/greeting from within the cluster? I will have another look at your samples in the upcoming days, if I get some time.

from spring-cloud-kubernetes.

@wind57

root@node1:~# curl 10.233.75.7:8888/actuator/gateway/routes
[{"predicate":"Paths: [/mockup/**], match trailing slash: true","route_id":"mockup","filters":[],"uri":"lb://mockup","order":0}]
root@node1:~#
root@node1:~#
root@node1:~# curl 10.233.75.7:8888/mockup/admin/greeting
{"timestamp":"2024-04-24T21:08:12.783+00:00","status":404,"error":"Not Found","path":"/mockup/admin/greeting"}
root@node1:~#
root@node1:~#

from spring-cloud-kubernetes.

You are calling the endpoint from the gateway. I said from the mockup ( on port 8761 ). Can you reach the mockup itself? Does that make sense? But still, something is off, you get a 404, not a 503. This means you cant even reach your gateway endpoint, to begin with. Didn't you have a context path root defined in gateway? Like /api/mockup....

from spring-cloud-kubernetes.

@wind57
Yes I can call directly gateway and mockup microservice from kubernetes CLI:

root@node1:~# curl 10.233.102.169:8761/admin/greeting
test response
root@node1:~#

But CLI -> gateway -> mockup is not working

I commended the /api context: https://github.com/rcbandit111/gateway/blob/main/src/main/resources/application.yml

from spring-cloud-kubernetes.

@wind57
I have an idea what might be wrong. When I deploy the projects via helm charts I use these configurations:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: role-full-access-to-secrets
rules:
  - apiGroups: [""]   # access secret - regcred
    resources: ["secrets"]
    resourceNames: ["regcred"]
    verbs: ["delete"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["create"]
  - apiGroups: [""]
    resources: ["nodes", "services", "pods", "endpoints"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["get"]
  - apiGroups: ["extensions"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch"]

Any idea which one of those I need?

from spring-cloud-kubernetes.

that's not it, those rules are only needed in order to be able to "get" the services. In other words, if those would not be correct, your gateway app would not even start.

The problem is elsewhere. I don't have the time in the upcoming days to figure it out, may be in a week or so, but not promising.

from spring-cloud-kubernetes.

I took your sample, and it works if I strip it down even more and add one thing to it:

spring:
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true

This is present in at least 4 places now that you are aware of :)

• the documentation
• the sample that I gave you (https://github.com/wind57/gateway-demo)
• indirectly discussed in the issue that I gave you : #775
• in your initial sample

from spring-cloud-kubernetes.

you can close the issue then...

from spring-cloud-kubernetes.