Giter VIP home page Giter VIP logo

cml.dev's Introduction

CML

CircleCI Link Check

CML project website's source code. Documentation content. Contributions are welcome!

Contributing Docs

Please see our Contributing guide for more details.

Running this site locally

git clone https://github.com/iterative/cml.dev
# Alternatively, save space with a shallow clone:
# git clone --depth 1 https://github.com/iterative/cml.dev
cd cml.dev
# Install dependencies
yarn

With the setup finished, you can now spin up the development server at http://localhost:8000 with yarn develop or try a full local build at http://localhost:9000 with yarn build && gatsby serve.

Getting help

If you have any questions, please join the community and use the #dev-docs channel to discuss any issues in our website or docs. We are very responsive and happy to help.

Copyright

Source code of this project is distributed under the Apache license version 2.0 (see the LICENSE file in the project root).

Except where otherwise noted, documentation and images are licensed under a CC BY 4.0 license.

By submitting a pull request for this project, you agree to license your contribution under the Apache license 2.0 (source code) or CC BY 4.0 (documentation). Exceptions could be made to content.

cml.dev's People

Contributors

0x2b3bfa0 avatar aguschin avatar casperdcl avatar charnysh-dev avatar daavoo avatar dacbd avatar danieljimeneznz avatar davidgortega avatar dependabot[bot] avatar dmpetrov avatar duijf avatar elleobrien avatar jendefig avatar jorgeorpinel avatar julieg18 avatar ludelafo avatar masanobbb avatar michaelgruner avatar nls-ybilleter avatar rcdewit avatar renovate[bot] avatar rogermparent avatar shcheklein avatar sisp avatar tadejsv avatar tasdomas avatar thomasjpfan avatar yathomasi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cml.dev's Issues

Multiple spaces in each line when "Copy" workflow yaml

When I copy the yaml and past it in vi/Notes it creates additional spaces in every single line like:

name: train-my-model



on: [push]



jobs:

  run:

    runs-on: [ubuntu-latest]

    container: docker://dvcorg/cml-py3:latest

    steps:

      - uses: actions/checkout@v2

      - name: cml_run

        env:

          repo_token: ${{ secrets.GITHUB_TOKEN }}

        run: |

          pip3 install -r requirements.txt

          python train.py



          cat metrics.txt >> report.md

          cml-publish confusion_matrix.png --md >> report.md

          cml-send-comment report.md


Fix IE11 / pre-chrome Edge support

This area will be updated as I figure out testing with these browsers, and what exactly is broken.

Depending on market share of incompatible browsers and the nature of required fixes, this could also be closed early. It's too soon to tell where it'll go.

contributing guide

Refers to 1. CML contribs as well as 2. cml.dev/doc contribs.

  • check dvc.org/doc guides for compatibility
  • also check cml wiki for special info
  • fix any issues
  • add cml.dev/doc nav entry redirecting to dvc.org/doc guide

Update option --token in cml-pr helppage

The helppage of cml-pr describes the option --token as a personal access token, but project access tokens can also be used for that. So i recommend to update the helppage by adding information about project access tokens.

Add 'How to install' section to the main page

Regarding the legal requirements, there must be a section on the website with a description of how to download and/or install the software.

Suggesting to add:

  • A ghost 'install' button in the header on the top right. A click will show a simple flyout dialog. It closes after clicking everywhere outside the menu or after pressing the 'OK' button. The command copy button would be very useful here.
    image

  • A button 'Download' in the Hero section. It should redirect to the 'https://github.com/iterative/cml#install-cml-as-a-package' page with an anchor, as well as link 'More in documentation' in the popover.

Design mockups โ†’ Open in Figma.

Full mockup example

When should we update dvc.lock and dvc push in GitHub Actions?

Discussed in iterative/dvc#6542

Originally posted by Hongbo-Miao September 6, 2021
Currently, my GitHub Actions workflow looks like this: when I open a pull request (change some model codes / params), CML creates a AWS EC2 instance, and DVC pull the data.

Here is my current GitHub Actions workflow:

Click to expand!
cml-cloud-set-up-cloud:
  name: CML (Cloud) - Set up cloud
  runs-on: ubuntu-20.04
  steps:
    - name: Cancel previous runs
      uses: styfle/[email protected]
      with:
        access_token: ${{ github.token }}
    - name: Checkout
      uses: actions/checkout@v2
    - name: Set up CML
      uses: iterative/setup-cml@v1
    - name: Set up cloud
      shell: bash
      env:
        REPO_TOKEN: ${{ secrets.CML_ACCESS_TOKEN }}
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
        AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      run: |
        cml-runner \
        --cloud=aws \
        --cloud-region=us-west-2 \
        --cloud-type=t2.small \
        --labels=cml-runner
cml-cloud-train:
  name: CML (Cloud) - Train
  needs: cml-cloud-set-up-cloud
  runs-on: [self-hosted, cml-runner]
  # container: docker://iterativeai/cml:0-dvc2-base1-gpu
  container: docker://iterativeai/cml:0-dvc2-base1
  steps:
    - name: Cancel previous runs
      uses: styfle/[email protected]
      with:
        access_token: ${{ github.token }}
    - name: Checkout
      uses: actions/checkout@v2
    - name: Set up Miniconda
      uses: conda-incubator/setup-miniconda@v2
      with:
        miniconda-version: "latest"
        activate-environment: hm-cnn
    - name: Install requirements
      working-directory: convolutional-neural-network
      shell: bash -l {0}
      run: |
        conda install pytorch torchvision torchaudio --channel=pytorch
        conda install pandas
        conda install tabulate
        pip install -r requirements.txt
    - name: Pull Data
      working-directory: convolutional-neural-network
      env:
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
        AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      run: |
        dvc pull
    - name: Train model
      working-directory: convolutional-neural-network
      shell: bash -l {0}
      env:
        WANDB_API_KEY: ${{ secrets.WANDB_API_KEY }}
      run: |
        dvc repro
    - name: Write CML report
      working-directory: convolutional-neural-network
      shell: bash -l {0}
      env:
        REPO_TOKEN: ${{ secrets.CML_ACCESS_TOKEN }}
      run: |
        echo "# CML (Cloud) Report" >> report.md
        echo "## Params" >> report.md
        cat output/reports/params.txt >> report.md
        cml-send-comment report.md

My dvc.yaml looks like this:

stages:
  prepare:
    cmd: tar -xf data/raw/cifar-10-python.tar.gz --dir=data/processed
    deps:
    - data/raw/cifar-10-python.tar.gz
    outs:
    - data/processed/cifar-10-batches-py/
  main:
    cmd: python main.py
    deps:
    - data/processed/cifar-10-batches-py/
    - evaluate.py
    - main.py
    - model/
    - train.py
    params:
    - lr
    - train.epochs
    outs:
    - output/models/model.pt

After training, if I think the change is good because the performance is better based on the reports,

  • the dvc.lock I feel needs to get update.
  • the new model model.pt needs to be uploaded to AWS S3 in my case.

My question is, after dvc repro, am I supposed to add dvc push and then commit? Something like

    - name: Train model
      working-directory: convolutional-neural-network
      shell: bash -l {0}
      env:
        WANDB_API_KEY: ${{ secrets.WANDB_API_KEY }}
      run: |
        dvc repro
        dvc push  # New added
        git add .  # New added
        git commit -m "update dvc.lock, etc."  # New added
        git push origin current_pr  # New added, need somehow get the current pull request name

This above method will apply when the pull request is open.
However, I kind of feeling the best moment adding would be when I decide merging because I think this is a good pull request that actually improves the machine learning performance. But at this moment, the EC2 instance has been destroyed.

Any suggestion? Thanks!

Special parameters for the git lab runner and docker container that it run

In some cases, I need to configure special parameters for the git lab runner that is created by the CML runner. I also need to be able to set special options for running the container.
For example:

  • git lab runner must have the parameter: privileged = true
    or
  • docker container must be started with the option: --log-driver=fluentd --log-opt fluentd-address=XX.XX.XX.XX:24224

Get rid of Netlify

Migrate to Heroku.
Netlify doesn't support CF caching on top of it, obscures too many things, doesn't give any reasonable value on top of Heroku + CF.

fix banner

There are a number of issues with the https://cml.dev banner

image

  1. "Install" should be replaced with "Get Started" linking to https://cml.dev/doc/start
  1. "Install" popup "More in documentation" links to non-existent https://github.com/iterative/cml#install-cml-as-a-package (can the link checker look for #ids?) Should be https://github.com/iterative/cml#local-package or https://cml.dev/doc/cml-with-npm
  2. the word "GitHub" should be replaced with "Source" or simply a GH icon as per https://dvc.org
  3. probably also add discord chat icon

Also "Download" button on landing page should be updated as per (1) above.

Replace Google Analytics

  • There are a couple ways we could do this. Two possibilities are using plausible analytics or have our backend handle analytics.

Update `Video` `play()`/`pause()` code, possibly clearing some Sentry errors

  • video.play() and video.pause() return a promise and we should place our code in something like a try and catch block. Not doing so may be causing some errors that were caught by Sentry
  • We could also use a catch block to take care of other Sentry video errors that we don't need to worry about

Links about issue

Sentry errors that could be resolved

Docs engine request

Implementation request for website team (if you're from CML or Documentation teams and want to discuss specifics please see #55 instead).

  • Can we have a new doc subdomain & page (the "Doc(s)" link in the hero/header should go to https://doc.cml.dev which should have a similar layout to https://dvc.org/doc)
  • In addition, we need subpages. All pages should have a ToC with a nested folder structure corresponding to layout in the cml-website repo.

Blank pages corresponding to folders:

  • doc.cml.dev/ CML Documentation
    • /start/ Getting Started
      • /start/github/ Getting Started with GitHub <-- sub-page
    • /ref/ Command Reference
      • /ref/#send-comment cml-send-comment <-- in-page <h2>s which show up in the ToC
      • ... etc ...

Each page can be blank for now (I'll fill in content) ๐Ÿ™ @rogermparent @julieg18

Code Tooltips

This is the last of what looked to me like intricate components on the homepage (other than the hamburger menu)

Highlights look like this:

image

And on mouseover a tooltip should pop up with this design:

image

The implementation shouldn't interfere with code copying, either through mouse or the copy button.

Add star indicator in the Hero section

Same as with dvc.org - it's important to have for two reasons:

  • trust - when people see 2000+ stars they trust the project more
  • more people will want to go to GH and be involved (including start it)

Update social sharing images

  • Currently all of our social images are using the old CML logo. We need to update all the images to have the new logo.

use tabs

use tabs where appropriate

follow-up to #113 <- #70

<toggle>
<tab title="GitHub">
foo bar
</tab>
<tab title="GitLab">
baz bat
</tab>
</toggle>

Update meta tag descriptions for docs pages

Currently we have a single description (summary about cml) for all pages. It would be better if we updated the descriptions depending on the docs page.

Possible option

  • Write a description for each doc index page and update any pages inside of it accordingly. For example, write a description for our "Get Started" section and give that description to the /start, /start/github, and /start/gitlab.

[discussion] New page(s): docs

Proposal. Discussion for @iterative/cml & @iterative/docs:

  • A new docs page at a subdomain (the "Docs" link in the cml.dev hero/header should go to https://doc(s).cml.dev which should have a similar layout to https://dvc.org/doc).
  • In addition, we need subpages. All pages should have a ToC.

Proposed layout:

  • doc(s).cml.dev/ CML Documentation
    • /features/ Features
    • /start/ Getting Started
      • /start/github/ Getting Started with GitHub
      • /start/gitlab/ Getting Started with GitLab
    • /guide/ User Guide
    • /ref/ Command Reference
      • /ref/#pr cml-pr
      • /ref/#send-comment cml-send-comment
      • /ref/#tensorboard-dev cml-tensorboard-dev
      • ... etc ...

Toggle lang button

Rather than writing separate pages for GH, GL and BB, it would in many cases be easier to write one page with a button to toggle between providers (similar to https://github.com/iterative/cml-website/blob/master/content/index.mdx but far simpler).

See e.g. https://zguide.zeromq.org/docs/chapter1/ for an example of clicking on a language next to code snippets to change the language of all snippets on the page (ofc we can do a lot better design-wise).

md interface options

html tags

# Installation

lorem ipsum...

<toggle>

<GitHub>

*bla bla* standard markdown...

```yml
- uses: iterative/setup-cml@v1
```

</GitHub>

<GitLab>

...

</GitLab>

<Bitbucket>

...

</Bitbucket>

</toggle>

tabbed

https://facelessuser.github.io/pymdown-extensions/extensions/tabbed/

=== "GitHub"

    *bla bla* standard markdown...

    ```yml
    - uses: iterative/setup-cml@v1
    ```

=== "GitLab"

    ...

other ideas?

URL interface options

  • cml.dev/doc/start?lang=gitlab#usage
  • other ideas?

/CC @iterative/cml @julieg18 @rogermparent

Getting ready for launch: Umbrella issue

This issue is meant to be a hub for discussion about pre-launch preparations for the cml.dev website. This issue should close just before the site goes public.

Sorted roughly in order of priority:

  • Highlighted text (tooltips part 1) #2 #3
  • Add final examples with tooltips in the right places (I think? may or may not remove cloud case)
  • SEO Part 1: Metadata and good Lighthouse scores #15
  • SEO part 2: Small details and keyword analysis #17
  • Make the final landing videos #9
  • Improve the container element for the landing video #4
  • Background image #10
  • Add the DVC.org logo to the footer
  • Add an action to the subscribe form #14
  • Ensure we have the social icons we want #13
  • Refine styles/sizes to be DRYer and fully match the Figma (Very close, working toward 100% if not there already)
  • Tooltips on highlights #2 #3 (Feature complete, but may need design adjustments)
  • Make a Readme for site contributors (A skeleton, currently)

Individual issues will be created and linked here. This umbrella issue is probably best used for discussion around discovering and adding new pre-release issues.

tabs: URL referencing

Given:

<!-- /some/page.md -->
<toggle>
<tab title="Foo">
</tab>
<tab title="Bar">

some text

### sub section

</tab>
</toggle>

Should be able to use:

  • /some/page?tab=Bar (or /some/page#Bar?) to auto-open the appropriate tab (p1-important, #130)
    • /some/page/?tab=Bar#sub-section (or /some/page#Bar-sub-section?) to auto-open the tab and scroll to containing section (p2-nice-to-have)
  • remember the ?tab=<arg> when navigating to a different page (p2-nice-to-have)

Other tasks:


thin font, low contrast

The paragraph under "CML Use Cases" uses thin-font and is on a very low contrast, which I'm finding very hard to read.

Or, is that the point?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.