Giter VIP home page Giter VIP logo

substra-frontend's Introduction



Substra


Substra is an open source federated learning (FL) software. It enables the training and validation of machine learning models on distributed datasets. It provides a flexible Python interface and a web application to run federated learning training at scale. This specific repository is the low-level Python library used to interact with a Substra network.

Substra's main usage is in production environments. It has already been deployed and used by hospitals and biotech companies (see the MELLODDY project for instance). Substra can also be used on a single machine to perform FL simulations and debug code.

Substra was originally developed by Owkin and is now hosted by the Linux Foundation for AI and Data. Today Owkin is the main contributor to Substra.

Join the discussion on Slack and subscribe here to our newsletter.

To start using Substra

Have a look at our documentation.

Try out our MNIST example.

Support

If you need support, please either raise an issue on Github or ask on Slack.

Contributing

Substra warmly welcomes any contribution. Feel free to fork the repo and create a pull request.

Setup

To setup the project in development mode, run:

pip install -e ".[dev]"

To run all tests, use the following command:

make test

Some of the tests require Docker running on your machine before running them.

Code formatting

You can opt into auto-formatting of code on pre-commit using Black.

This relies on hooks managed by pre-commit, which you can set up as follows.

Install pre-commit, then run:

pre-commit install

Documentation generation

To generate the command line interface documentation, sdk and schemas documentation, the python version must be 3.8. Run the following command:

make doc

Documentation will be available in the references/ directory.

Changelog generation

The changelog is managed with towncrier. To add a new entry in the changelog, add a file in the changes folder. The file name should have the following structure: <unique_id>.<change_type>. The unique_id is a unique identifier, we currently use the PR number. The change_type can be of the following types: added, changed, removed, fixed.

To generate the changelog (for example during a release), use the following command (you must have the dev dependencies installed):

towncrier build --version=<x.y.z>

You can use the --draft option to see what would be generated without actually writing to the changelog (and without removing the fragments).

substra-frontend's People

Contributors

acellard avatar alexisdeh avatar aureliengasser avatar clairephi avatar dependabot[bot] avatar guilhem-barthes avatar hamdyd avatar ic-1101asterisk avatar jmorel avatar kelvin-m avatar maeldebon avatar mblottiere avatar milouu avatar oleobal avatar romaingoussault avatar romainlq avatar samlesu avatar sdgjlbl avatar sergebouchut2 avatar thbcmlowk avatar thibaultfy avatar vantiennguyenowkin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

substra-frontend's Issues

Refactor Search cpt componentDidMount method

By the way, I think the search component should not call its componentDidMount method in the first place.
I assumed the componentDidMount of the search component is called whenever with switch to a new route. But in reality, we should handle it ourself. Maybe using a redux action.

For now removing the plugin fixes the issue in production. This is a hotfix.

Originally posted by @GuillaumeCisco in https://github.com/SubstraFoundation/substrafront/issues/70#issuecomment-457126825

How to run the frontend using the docker environment.

I would like to install the frontend code inside the docker, and run it using docker-compose.
But I found some requirements in the docker-compose.yaml file.

"environment:
- API_URL=$API_URL
- NODE_PORT=$NODE_PORT
- SECURE_NODE_PORT=$SECURE_NODE_PORT "

How can I set the $API_URL, $NODE_PORT and $SECURE_NODE_PORT?

Thank you.

Cleanup readme

The current readme seems wildly out of date with how we develop and deploy the frontend:

  • it doesn't mention skaffold
  • it doesn't explain how to use yarn start with a backend deployed in k8s
  • etc.

It should be reviewed thoroughly.

Refacto + New Features Search Bar

  • Refacto code for not using material-ui components nor downshift.
  • Add AND NOT filter
  • Review First/Second options (maybe put directly challenge:name in the first menu)
  • Use base64 in url and decode it inside backend
  • Fix css on small screens
  • Automagically select right filter when posting uuid directly (May be impossible due to blockchain philosophy)

Bug: update a user after trying to update the last existing admin

Issue description

I found a small issue in the user management page of the frontend where you can't change the role of a user if you tried to change the role of the last admin right before.

screen.record.mov

Additional info

Frontend commit: 4a7d338
It happened on a fresh platform deployed locally with no data and a single additional user created through the front.

[maintenance] Merge substra-ui back into substra-frontend.

The substra-ui library was supposed to allow the creation of new frontends for substra but no such project has seen the light and the extra dependance makes substra-frontend harder to maintain. We should really merge substra-ui back into substra-frontend and close substra-ui repository.

Here is the proposed migration plan.

1st step is to migrate all files:

  • all components go directly under src/app/business/common/components
  • globalStyles: lato goes under assets, the globalStyle component goes into src/app/business/common/components
  • icons go under src/app/business/common/components in a common folder
  • utils/propTypes.js goes under src/app/utils
  • variables go under assets and are to be merged with existing variables

2nd step is obviously to update all references to these new files instead of substra-ui (and remove all mentions of substra-ui from the codebase).

3rd step is to include in substra-frontend the storybook and jest configurations so that all existing tests are ported to substra-frontend as well.

skaffold dev failed with Error: uninstall: Release not loaded: frontend-org-1: release: not found

I set frontend-org-1 and frontend-org-2 to 127.0.0.1 (I use microk8s), and ran skaffold dev.

The error is
Release not loaded: frontend-org-1: release: not found

Build Failed. No push access to specified image repository. Try running with --default-repo flag. Otherwise start a local kubernetes cluster like minikube

I don't see any instructions on how to create this "release". Please provide a hint.

Update browse related dataset wording for algorithm

Algorithms aren't directly related to datasets. The only relationship they have is through the challenge (test dataset) or through the traintuples. As a result, a standard "datasets" link in the browse related section has no meaning.

We should rename this link into "test dataset" since this is what is returned by the /dataset endpoint when there is a filter on an algorithm applied.

We should also rename the "dataset" link on the challenge page into "test dataset" to make the relationship more explicit.

Filter model buttons do not work

Go to the models section, click on either the "filter" button in the more menu in the list or on the filter button in the detail pane and you'll get no result.

This is because the hash pushed to the searchbar is the traintuples and not the models.

We could fix it easily but then it begs the question of failed traintuples: since these don't have a model, they also don't have a hash we can push to the searchbar.

I see two alternatives:

  1. Do not display filter buttons for failed traintuples
  2. Add support for filtering on traintuple keys and not just model keys. Then filter failed traintuples on those keys.

[Bug] Error when clicking on a composite traintuple with status failed

Setup

Local deployement of Substra using skaffold
Latest commit on master (tag: 0.0.17)

Description of the bug

On the models page: http://substra-frontend.node-1.com/model
I have a composite traintuple with the status 'failed'. When trying to click on it, the page becomes entirely blank.

Opening the dev tools gives me this error:

react-12ba27.js:2 TypeError: Cannot read property 'hash' of null

This happens with two different failed composite traintuples, no problem with failed traintuples and aggregatetuples, nor with 'done' tuple.

The error seems to come from this:
image

In case of failure, outHeadModel.outModel and outTrunkModel.outModel are null instead of something like this: outModel: {hash: "77ac6ae57ef74bc465a676e584c71a4d6f0ae083a5bb29e4a532a38e77355732"}

Tests

  • Actions
  • Sagas
  • Reducers
  • Components with Jest
  • Apis
  • Covering

GDPR compliancy

Since we now have Google Analytics installed on the frontend, we should probably investigate what measures must be taken to comply with the GDPR.

NB: the platform is currently a private, internal test, but we should be prepared for the future.

Cannot visualize dataset informations in some deployed frontends behind proxys

After registering a new dataset to be public, the dataset informations (description/openers) are not visible on some frontends deployed behind proxys.

Inspecting one of the faulty frontend webpage gives this

main-15d707.js:1 Mixed Content: The page at 'https://front-mydeployedsubstra/dataset' was loaded over HTTPS, but requested an insecure resource 'http://api-deployedfrontend/data_manager/longhash/description/'. This request has been blocked; the content must be served over HTTPS.

Seems like url rewrite of assets is mistaken on the http/https.
All front execute a request using:

request.build_absolute_uri(
        reverse('substrapp:data_manager-description', args=[data_manager['key']]))

Django doc confirms that http/https depends on the request format:

Maybe it should be based on the headers PROXY ?

"add testtuple" command are invalid

When a traintuple has no testtuple associated, the frontend offers a "add testtuple" command. Since the removal of objectives from traintuples, this command is missing the --objective-key parameter and therefore not working.

Since we cannot infer what the objective-key should be, the commands should be removed entirely.

Helm chart incompatibility with latest redis chart release

👋 The helm chart is currently broken du to a change on the redis chart dependency.
I suspect they adapted the chart to work with Helm 3 but made the chart unsuitable for Helm 2 when used as a dependency. It's to be noted that installing the redis chart alone works fine so it have to be related to chart inheritence. Also the stable/redis chart is now deprecated and is to be replaced by the bitnami one.

How to reproduce

$ helm fetch substra/substra-frontend --version 1.0.0-alpha.2
$ helm template substra-frontend-1.0.0-alpha.2.tgz 
Error: render error in "substra-frontend/charts/redis/templates/redis-master-statefulset.yaml": template: substra-frontend/charts/redis/templates/redis-master-statefulset.yaml:81:18: executing "substra-frontend/charts/redis/templates/redis-master-statefulset.yaml" at <eq .Values.securityContext.runAsUser 0.0>: error calling eq: incompatible types for comparison

Workaroud

As of today you need to disable the dependency using the flag redis.enabled and install it yourself.

Debug infos

$ helm version
Client: &version.Version{SemVer:"v2.15.0", GitCommit:"c2440264ca6c078a06e088a838b0476d2fc14750", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.15.0", GitCommit:"c2440264ca6c078a06e088a838b0476d2fc14750", GitTreeState:"clean"}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.