Ordering on xDS responses about go-control-plane HOT 5 CLOSED

@kyessenov Hi Kuat!

May I wonder why this issue hasn't got any attention since it was opened ? Isn't it a well-known issue that is sitting there to happen ?

For the context, I've recently described the issue I'm facing on #control-plane-dev Slack:

I’ve run into an issue where envoy keeps updated cluster in a warming state for indefinite amount of time.

As far as I understand, such behaviour is explained in the Resource warming section of the xDS Protocol (https://www.envoyproxy.io/docs/envoy/latest/api-docs/xds_protocol#resource-warming):

Clusters and Listeners go through warming before they can serve requests. This process happens both during Envoy initialization and when the Cluster or Listener is updated. Warming of Cluster is completed only when a ClusterLoadAssignment response is supplied by management server. … Management server is expected to provide the EDS/RDS updates during warming. If management server does not provide EDS/RDS responses, Envoy will not initialize itself during the initialization phase and the updates sent via CDS/LDS will not take effect until EDS/RDS responses are supplied.

In my case, while Cluster definition is updated, endpoints do not change. That’s why my go-control-plane-based control plane doesn’t send a response to EDS request.

Am I interpreting envoy behaviour correctly ? Shouldn’t this case be handled by go-control-plane ?

from go-control-plane.

You should make sure the snapshot is consistent if you use ADS mode. That means every cluster has an endpoint assignment (there is a helper function Consistent() to help with that).

Keep in mind, that most production xDS servers are only eventually consistent. That means they generally expect EDS server to catch up to CDS server and tolerate partially broken config in envoys (which can sometimes lead to 5xx/4xx). You can simulate that behavior in go-control-plane by not using ADS mode.

The entire sequencing is complex, so I simply didn't have time to do it right. It appears folks are fine with eventual consistency for now.

from go-control-plane.

Thank you!

from go-control-plane.

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

from go-control-plane.

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.

from go-control-plane.