flocker-reportstate: report local state of a node about flocker HOT 10 CLOSED

It's possible that for the initial set of functionality we're targetting, each node will only need to know its own current configuration and the desired new configuration. That is, we may not need to aggregate all of this and publish the result to all nodes. Nodes may get by just producing and consuming this information locally.

from flocker.

We decided two-pass is necessary for volume management. So we do need this tool, as part of two phase deploy.

The command line tool would run flocker.node.Deployer().discover_node_configuration() and serialize it to YAML utilizing approximately the same structure as the application and deployment configs. Initially some information will not be included (e.g. #207 is necessary for images), but that's OK for now.

from flocker.

Note that Tom had a proposed alternative to this (nodes wait until all handoffs are done, then create whatever is still non-existent) so we should consider that before proceeding with this.

from flocker.

I think that approach requires more direct node-to-node communication. Without such, nodes don't have any way to know when all the handoffs are done. I think it will be easier not to have to introduce that communication.

from flocker.

At least a start of this can be made even without #271.

from flocker.

Taking a stab at this.

from flocker.

I've started something here; still not a full sketch:
https://github.com/ClusterHQ/flocker/compare/reportstate-11

flocker-reportstate (or maybe flocker-node reportstate, see below) would create a Deployer instance, calculate the local node state and use the new function to serialize that to YAML.

The assumption is that we can reuse (with a tweak or two) the application configuration parser to read this format; the semantic meaning is different (application that can be enabled vs applications on a particular node) but the contents should be the same.

Open question: New command line program, or turn flocker-changestate into flocker-node changestate?

from flocker.

@itamarst

@dwgebler and I updated your sketch including some details of how we intend to hook up the new configuration function to a flocker-reportstate script.

We also added some proposed tests for the configuration function and for the script.

We decided that it'll be easier (for now) to create a flocker-reportstate script rather than subcommand of flocker-node. We can arrange it that way in the future if necessary.

Not sure if we need to write any functional tests for the new script.

Also not sure if it's worth writing some end-to-end tests ie configuration_to_yaml > _applications_from_configuration and asserting that the resulting applications are the same as those we supplied.

If you approve we'll continue along these lines this afternoon.

from flocker.

Thanks. Some minor points:

1. main needs to write the yaml to stdout, not just return a Deferred that fires with it
2. safe_dump instead of dump (because why make safety the default ...)
3. It won't be possible to make all of the sketched tests pass. Specifically, without Deployer actually being able to discover mountpoint information, I don't think test_application_with_volume_includes_mountpoint can work.

Otherwise seems fine. Please proceed.

from flocker.

Not sure if we need to write any functional tests for the new script.

To demonstrate setup.py has defined it correctly, at least, yes.

Also not sure if it's worth writing some end-to-end tests ie configuration_to_yaml > _applications_from_configuration

Eh. This might not hurt but it also doesn't seem strictly necessary. Use your judgement as your proceed with the implementation, I'd say.

from flocker.