netflix / metaflow-docs Goto Github PK

Ensure that the options in https://github.com/Netflix/metaflow/blob/master/metaflow/metaflow_config.py are part of the API docs.

Although available through the Cloud Formation template, the manual deployment guide does not mention the Fargate options (Fargate, Fargate Spot) when creating compute environments for AWS Batch. Given the excellent quality and level of detail of the guide, this should be added.

cc: @obgibson

In the Data Flow Through Graph section of the Basics of Metaflow page in the documents, the example is broken. When it is run it produces the error:

Step join is both a join step (it takes an extra argument) and a split step (it transitions to multiple steps). This is not allowed. Add a new step so that split and join become separate steps.

The link https://lh5.googleusercontent.com/f68UAR7UxClKR6XvHXq-ZeepDXLHISoVboj6amF9f0cv6jCxqTsjyYnm-isKIHdINMl\_gdlPVbR3wJ9vIDwipjMIglHwwetL3-rVNhUTRN4a7UYXmVZ78IqeI2fGJ2OA2Silk5ZS is broken in the
What is Metaflow section of the docs, which would depict the layers of the data science infrastructure stack.

It would be nice to see the intended image.

Please hide the import statement if show_import="False"

<DocSection type="property" name="input" module="metaflow.flowspec" show_import="False" heading_level="4" link="https://github.com/Netflix/metaflow/tree/master/">
<Description summary="A Flow represents all existing flows with a certain name, in other words,\nclasses derived from 'FlowSpec'" extended_summary="As such, it contains all Runs (executions of a flow) related to this flow." />
</DocSection>

@obgibson

The link in the very bottom paragraph of the AWS CloudFormation Deployment guide should link to https://admin-docs.metaflow.org/overview/configuring-metaflow, but actually links to https://admin-docs.metaflow.org/metaflow-on-aws/deployment-guide#configuring-metaflow

Once you have followed all these steps, you can configure your metaflow installation using the outputs from the CloudFormation stack.

cc: @obgibson

On the page "What is MetaFlow" is a link to a youtube playlist "Overviews of Metaflow".
This link leads to an error saying that the playlist doesn't exist.
Same issue for the link "Internals of MetaFlow"

See error message

We would like to have Anchor Tags in the documentation so we can link to specific sections, this is especially helpful for API docs when there are many sections on a page.

cc: @obgibson

On page https://docs.metaflow.org/metaflow-on-aws the links at the bottom of the page, pointing to "deployment guide" and "operations guide", are broken.

This https://outerbounds.com/docs/aws-deployment-guide looks like the place for the deployment guide but I'm not sure where the operations guide should point.

https://outerboundsco.slack.com/archives/C031ZJA4ZBR/p1655761217509929?thread_ts=1655760716.938469&cid=C031ZJA4ZBR

cc: @obgibson

The docs and templates talk about METAFLOW_SERVICE_URL in a few places, which seems self explanatory. But they don't really mention that METAFLOW_DEFAULT_METADATA also needs to be set to use remote metadata service. Took me a while to figure this out, maybe there is a way to make it more obvious in the docs?

https://numpy.org/doc/1.22/reference/generated/numpy.ndarray.size.html

cc: @obgibson

First of all, thank for creating such an elaborate documentation for Metaflow in general and for manual deployment specifically 👍🏻. I assume it can be tedious to keep it up to date, but it really helps users who can not use the Cloud Formation template for various reasons (such as myself).

While reading through the manual deployment guide within the admin docs section, I noticed that you refer to using an existing Elastic IP (Compute > Create VPC > 1. Run VPC Wizard > Step 4) , but do not mention how to create it in the first place:

For Elastic IP Allocation ID, choose the ID of the Elastic IP address that you created earlier.

While readers may figure out how to do so, it would be great if it was mentioned in the guide directly.

The resume command is a powerful one but a lot of people only consider it when something failed (if then). resume, however, can be used for other things and not necessarily when things have failed (you may want, for example, to reuse some initial steps and change the end). Improving documentation around these use cases may be useful to users.

The following Doesn't render a the description

<DocSection type="property" name="input" module="metaflow.flowspec" show_import="False" heading_level="4" link="https://github.com/Netflix/metaflow/tree/master/">
<Description summary="A Flow represents all existing flows with a certain name, in other words,\nclasses derived from 'FlowSpec'" extended_summary="As such, it contains all Runs (executions of a flow) related to this flow." />
</DocSection>

Results in

It appears that if I add a SigArg component the description renders

<SigArgSection>
</SigArgSection>

But this is not desired behavior, since the presence of SigArg also adds parenthesis to the rendered object name.

cc: @obgibson