When installing ray operator with the following settings

singleNamespaceInstall: true
rbacEnable: true
crNamespacedRbacEnable: true
watchNamespace:
- xxx

we get the error

E1122 16:36:19.205328       1 reflector.go:138] go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:167: Failed to watch *v1.Job: failed to list *v1.Job: jobs.batch is forbidden: User "system:serviceaccount:default:kuberay-operator" cannot list resource "jobs" in API group "batch" in the namespace "xxx"

I believe this is due to missing rules from
https://github.com/ray-project/kuberay-helm/blob/main/helm-chart/kuberay-operator/templates/multiple_namespaces_role.yaml

The batch.jobs RBAC permission is present in the cluster role equivalent here

Add charts for the KubeRay 0.4.0 release.

cc @pcmoritz @Jeffwan, FYI on the existence of this repo.

If we install apiserver, operator in namespace X and operator with watchNamespac [X, Y], trying to manage CRD in Y, the setup leads to

2023/11/22 16:56:17 could not list clusters rpc error: code = Unknown desc = List clusters failed.: List RayCluster failed in tsray: rayclusters.ray.io is forbidden: User "system:serviceaccount:default:kuberay-apiserver" cannot list resource "rayclusters" in API group "ray.io" in the namespace "Y"

Should we consider also installing the RBAC for CRD in kuberay-apiserver to a list of watchNamespace like kuberay-operator?

I'm happy to provide that PR if maintainers believe this is a reasonable request

I'm playing around with the combination of Kuberay and Kueue for quota management. In order to have Kueue manage the resource quotas for RayClusters, I need to put an additional label on the CR (not the head/worker pods; see the linked docs).

The current Helm chart unfortunately does not expose a way to do this easily, so I have to resort to helm template and manually add the label in the rendered manifest (I can't do kubectl label, either, since the admission controller apparently does not allow modifying the labels on an existing RayCluster CR).

Would it be possible to add a way in the Helm chart to pass extra labels to be applied to the RayCluster CR via values.yaml? I can provide a PR, if you are interested.