datmo / datmo Goto Github PK

On ubuntu,

ubuntu@ip-172-31-24-161:~/datmo$ datmo -v
usage: 
        
        datmo COMMAND [SUBCOMMANDS] ARGS 

        Datmo is a command line utility to enable tracking of data science projects. 
        It uses many of the tools you are already familiar with and combines them into a snapshot
        which allows you to keep track of 5 components at once

        1) Source Code
        2) Dependency Environment
        3) Large Files
        4) Configurations
        5) Metrics
        
        command: 
        
datmo: error: too few arguments

https://github.com/datmo/datmo/blob/master/datmo/cli/command/snapshot.py#L97

if arg in kwargs and kwargs[arg]:

This line will return false if the kwargs[arg] value is falsey.

https://docs.python.org/2.4/lib/truth.html

Maybe it should be kwargs[arg] != None , but can None be a valid param sometimes?

• Enforce Immutability of Objects and/or immutability of object attributes
• Validation for entities using the schemas.yml

https://app.codacy.com/app/datmo/datmo/dashboard

• Add Contributer.md with instructions for contributers and move current developer materials in README.md to CONTRIBUTER.md or another file
• add badge for pypi into readme : https://shields.io/

Instead of iris_sklearn.py

Use names like sdk_create_snapshot.py ,cli_datmo_init.py

add code to automatically register the environment of the snapshot.

Can blitzDB be pushed to source control? is there any binary data? Merge conflict resolution?

• Describe syncing issues of BlitzDB in readme

• simple interactive mode scripts, e.g. run a bash terminal, or a python terminal with task run
  datmo task run "bash" --interactive, datmo task run "python" --interactive
• short running scripts where you want to see the output immediately on the stdout
  datmo task run "python script.py", datmo task run "jupyter notebook" --ports 8888:8888

• tasks
• snapshots

• edit code to leverage entity properties

https://github.com/datmo/datmo/blob/master/datmo/cli/command/snapshot.py#L65

Also, this function can be helpful on other commands, so it should be moved to a utils class.

• Throw exception if both args exist
• Move to util class so other commands can use it

I believe this can be within each function because each function can tell if the inputs it receives is valid or not.

• move to environment and remove from task
• simplify snapshot to not pass other controller inputs
• simplify task to not pass other controller inputs

Create a global home property, that can be used everywhere instead of having to pass home down to all components.

• change README.md to match website
• add CHANGELOG or HISTORY document for public releases
• set versioning conventions and document and add to Developer Notes / Tools
• instructions for a release in the Developer Notes / Tools section

• Datmo installation, datmo init, then copy paste a small snippet of code into a file like "train.py", then datmo run and datmo snapshot, and view results in 2 min.

• One person creates a project and a snapshot (see 1) , then the project is pushed onto github. Then another user clones from github, then the other user views the snapshots, then the other user creates snapshot on their system, then the first person pulls the changes from the other user, then the first user is able to see all of the snapshots from themselves and the other user

• Controllers
• Abstract drivers

• add more info into docstrings and standardize (datmo.cli (pulled from argparse object), datmo.snapshot, datmo.task, datmo.init)
• Convert REAMDE from md to rst and clean up the code example

Since, the sdk gives back the results, it makes sense to have CLI to provide it in datmo task ls

create python module for easy importing. should work for the following snippet (https://github.com/datmo/example-code/blob/master/hello_world_iris_datmo.py)

• clean up sphinx (ignore unnecessary files)
• clean up indices

• directory within examples/
• includes clear instructions for how to use the files and test
• enables user to see snapshots and run tasks

• check if blitzdb file backend works to sync
• git refs should not be in /refs/datmo (make this a config)

• Adding query dict to controller to be able to get relevant rows
• Adding time query to sdk, cli code and list them