Giter VIP home page Giter VIP logo

jina-ai / dashboard Goto Github PK

View Code? Open in Web Editor NEW
117.0 35.0 60.0 134.36 MB

Interactive UI for analyzing Jina logs, designing Flows and viewing Hub images

Home Page: https://dashboard.jina.ai

License: Apache License 2.0

JavaScript 3.60% HTML 0.43% CSS 2.28% TypeScript 93.63% Shell 0.06%
jina neural-search dashboard flowchart jina-search jina-hub containerization reactjs hub log-streaming registry

dashboard's Introduction

dashboard logo

CD Jina Jina License Jina Docs We are hiring tweet button

Dashboard allows you to build Jina Flows using a graphical interface. Drag and drop your pods quickly; no code required! Run Flows and monitor by exploring detailed Logs!

Jina Dashboard is a low-code monitoring and management environment for Jina. With the Dashboard, you can build Jina Flows and manage them from a central location. Get detailed insights into the health of the Flow with the use of log stream analysis!

To use the Jina hosted version of the Dashboard visit the Jina website

log server settings

🌟 Features:

Flow Design GUI

Use Jina easily in the browser; no code required! Building flows within the user-friendly interface is easy, just start dragging pods into the design canvas. View the list of Pod properties and edit them using a visual menu. To create a custom Flow: connect Pods, upload a YAML file, or use an existing Flow template!

log server settings

LogStream and Task Monitoring

Dig deeper into your Jina Flow using the LogStream, and debug your Flow by viewing your logs in real-time! Jina makes it extremely easy to stay on top of your search deployment by allowing you to filter, group, and search logs based on Pod, log-level, or message!

log server settings

Hub Integration

Browse Hub images uploaded by other users! Search and filter images according to various tags. Identify all information needed to use the Hub Pods with ease. All you need is a simple Docker pull command. Embrace the power of Open Source and a community-driven codebase!

log server settings

Contents

Getting started

1. Start Jina Daemon

In order to use the dashboard you will need to start a Jina Deamon. You can do this by either using a Docker container that has been pre packaged for you or manually using a Command Line Interface (CLI).

Using Jina Daemon with Docker (Reccommended)

  • Install Docker and run Docker Desktop
  • Pull the Docker image with
docker pull jinaai/jina:master-daemon
  • Run the Docker image with
docker run -p 8000:8000 -p 5555:5555 jinaai/jina:master-daemon

log server settings

Using Jina Daemon with Command Line Interface (Not reccomended)

  • Install JinaD
  • Run JinaD with
jinad

log server settings

2. Connect the Dashboard to Jina Daemon

  • Go to Settings
  • Set host to: http://localhost
  • Set port to: the port specified when running JinaD (8000 in this example)
  • Save changes
  • Click 'Refresh' in the bottom right corner

log server settings

  • The globe icon in the top right corner will indicate connection status. A red X means the Dashboard is not connected to any Jina instance; a green checkmark indicates an active connection. The icon should change to green to show Dashboard, and the Jina Deamon are now connected.

log server settings

  • You should now be able to see the log stream and flow visualization.

Self-hosted Dashboard

You can also use a self-hosted dashboard instance locally.

Note: features like the Hub will not work when running the Dashboard locally. They are restricted to the dashboard.jina.ai origin. They are not necessary to view logs or interact with flows. If you would like to browse the Hub, do so from dashboard.jina.ai.

Set up

  • Clone the GitHub repo
git clone https://github.com/jina-ai/dashboard.git && cd dashboard
  • Install dependencies using command
npm i

Run dashboard

Run in debug mode

  • Start the testServer
npm run start_dev-server
  • testServer should now be running on http://localhost:5000 by default

  • Start the dashboard

npm run dashboard start
  • Dashboard should now be served on http://localhost:3000 by default

Run in the live mode

  • Build dashboard:
npm run dashboard build
  • Run dashboard:
node dashboard
  • Dashboard should now be served on http://localhost:3030 by default

Contributing

We welcome all kinds of contributions from the open-source community, individuals and partners. Without your active involvement, Jina can't be successful.

The following resources help you to make a good first contribution:

Community

  • Code of conduct - play nicely with the Jina community
  • Slack workspace - join #general on our Slack to meet the team and ask questions
  • YouTube channel - subscribe to the latest video tutorials, release demos, webinars and presentations.
  • LinkedIn - get to know Jina AI as a company and find job opportunities
  • Twitter Follow - follow and interact with us using hashtag #JinaSearch
  • Company - know more about our company and how we are fully committed to open-source.

Open Governance

As part of our open governance model, we host Jina's Engineering All Hands in public. This Zoom meeting recurs monthly on the second Tuesday of each month, at 14:00-15:30 (CET). Everyone can join in via the following calendar invite.

The meeting will also be live-streamed and later published to our YouTube channel.

Join Us

Jina is an open-source project. We are hiring full-stack developers, evangelists, and PMs to build the next neural search ecosystem in open source.

License

Copyright (c) 2020-2021 Jina AI Limited. All rights reserved.

Jina is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

dashboard's People

Contributors

aga11313 avatar alexcg1 avatar bastinjafari avatar cpooley avatar dalekatwork avatar davidli-oneflick avatar davidsanwald avatar deepankarm avatar dependabot[bot] avatar fayeah avatar florian-hoenicke avatar franquil avatar hanxiao avatar helaoutar avatar imsergiy avatar jina-bot avatar lusloher avatar mkhilai avatar nan-wang avatar pdaryamane avatar prasanth94 avatar rjgallego avatar roshanjossey avatar saurabhnemade avatar seraco avatar showtim3 avatar sulemanbhatti avatar xiqi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dashboard's Issues

redesign the 404 page

404 page should contains a simple cards points to the link, icon use font awesome

image

TypeError: _store.summaryCharts[level] is undefined

Hi, thanks for the fabulous Jina!

TL;DR:
When i try to access them logs on the dashboard i get the following error:

grafik

More Details:

Self-hosted a dashboard and it worked out fine (the pod_api line for mylogserver.yml is not yet in the docs). The dashboard is hosted on a remote machine and i can access from my local machine. So far so good.

grafik

On the same remote machine i am indexing the southpark example. I edited flow-index.yml to use logserver.
root@tmk-0-fra1:~/ex2/examples/southpark-search# cat flow-index.yml:

!Flow
with:
  read_only: true
  port_expose: $JINA_PORT
  logserver: true
  logserver_config: mylogserver.yml
pods:
  reader:
    yaml_path: pods/extract.yml
  splittor:
    yaml_path: pods/craft-split.yml
    replicas: $REPLICAS
    read_only: true
  encoder:
    yaml_path: pods/encode.yml
    replicas: $REPLICAS
    timeout_ready: 1200000
    read_only: true
  chunk_indexer:
    yaml_path: pods/index-chunk.yml
    replicas: $SHARDS
    separated_workspace: true
  doc_indexer:
    yaml_path: pods/index-doc.yml
    needs: gateway
  join_all:
    yaml_path: _merge
    needs: [doc_indexer, chunk_indexer]
    read_only: true

root@tmk-0-fra1:~/ex2/examples/southpark-search# cat mylogserver.yml:

host: 0.0.0.0
port: 5000
endpoints:
  log: /stream/log  # fetching log in SSE stream
  profile: /stream/profile  # fetching profiling log in SSE stream
  yaml: /data/yaml  # get the YAML spec of a flow
  shutdown: /action/shutdown  # shutdown the log server
  ready: /status/ready  # tell if the log server is ready, return 200 if yes
  podapi: /data/api/pod  # get the api spec of the pod

run python3 app.py -t index -n 250
grafik

But when i try to access them logs it gives a bunch of erros (they're all the same):
grafik

Flow state not being saved to local storage

Describe the bug
When playing with flows (adding/deleting) the state of the flows should persist the next time you refresh. Flows should be saved to local storage with relevant data (except for things like connection status, jinad flow_id etc)

broken task page view for some screen sizes

As it can be seen in the next screenshots (768 x 1024 pixels and 320 x 568 pixels respectively), the home page view is broken for certain screen sizes:

Screenshot 2020-12-08 201103
Screenshot 2020-12-08 201213

The following issues are observed:

  • Progress chart falls outside of its card in some screens.
  • Network Load chart does not properly display for certain screen sizes.

Authentication issue when self hosting the dashboard

Hi @hanxiao,
I am new to jina-ai but I decided to spend more time to play with it and hopefully contribute with PRs(appreciate any recommendation about which repo to start).
I have tried to host the dashboard on my localhost following the doc but I am having an authentication issue.
I followed the code and it looks like the github authentication goes through jina hub url.
And when it returns successfully it redirects to jina ai dashboard instead of localhost dashboard.

So I have a few questions:

  • Why is the authentication is part of hub but not dashboard if this is a standalone project?
  • Should I also run hub locally to be able to host the dashboard?(I will try this after posting this but writing here on parallel maybe I can gain some time:))

Or am I missing something?
Thanks

Put react-flow-chart-withtooltips-and-multi-select under Jina's namespace

I published an npm package under my user name when I implemented tooltips and multi-select in order to implement both into the flowcharts. It would be better to put them, as long as the pull request into the official package doesn't go anywhere. under jinas namespace. That way we can maintain it as a company

Decompose Settings .jsx into small components

I've checked Settings.js which supposed to be ( maybe) SettingsCard as a name of the file.

I find that component is complex and it does many things and I suggest decomposing it into small components that do one thing ( SRP) ( Header, Body, Footer) because it's a Card. This will make scaling easy in the future and will help you maintain the Unit Test in decomposable files and in Isolation.

βš—οΈ Create integration tests

Problem

Currently, this project has jest setup to do unit tests but there is no tool to do integration tests.

Goal

Have integration, end-to-end tests to make sure that the components work together. This will give us more confidence in releasing. We are now using cypress for integration tests.

Next steps
We'll create issues to help community tackle this. It would be a good way for new people to start engaging with this project too.
I'll maintain a list of issues we create on this topic.

broken home page view for some screen sizes

As it can be seen in the next screenshot (768 x 1024 pixels), the home page view is broken for certain screen sizes:

Screenshot 2020-11-29 123451

The following issues are observed:

The reason why this might be happening is because there might be not enough horizontal space for displaying all the cards that are being displayed.

Pods increasing in size

Describe the bug

Sometimes the pods in FlowView increase in size infinitely.

Screen.Recording.2021-01-29.at.10.35.37.mov

πŸ§ͺ Create integration tests for help view

πŸ§ͺ This is the view that should be tested

Screenshot from 2021-01-12 12-41-23

πŸ’» How to run this project locally

  1. Fork this repository by clicking on fork button on the top right. This will create a copy of this repository on your account.

  2. Clone your fork.
    Get the URL of your fork by clicking on code button on GitHub.
    Clone it to your local with command git clone <URL you copied>

    git clone <https://github.com/><your-username>/dashboard.git.

  3. Install dependencies using command npm i.

cd dashboard
npm i
  1. Run dashboard via the following ways .

Run in the debug mode

  1. npm run start_dev-server

    testServer will be running on http://localhost:5000 by default

  2. npm run start-dashboard

    Dashboard will be served on http://localhost:3000 by default

πŸƒβ€β™€οΈ How to get there.

Click on the option Help from the sidebar on the left

The component of interest are NotFound

πŸ§ͺ How to run integration tests

Run npm run cy:dev from your console. This will open cypress and you can run the test suites you want to run by clicking on it.

When you're done working, make sure that npm run cy:run passes without any errors.

πŸ“ Where to make changes?

In cypress/integration/views directory and in the components you want to test.
Check this pull request

Pay attention to the changes in files cypress/integration/views/logsView.ts and src/components/LogStream/BarChartBase.tsx. You'll have to make similar changes.

πŸ“– How to learn about writing integration tests?

We use a framework called cypress for integration tests. Their documentation is quite helpful.
Go through their best practices, interacting with elements, structuring tests and you'll get a great understanding of how to write good integration tests

πŸ–οΈ I want to work on this.

If you're interested in working on this issue, please write a comment down below. πŸ’¬

If you need any help or have questions, you can ask questions here or ask in our slack group

start-dashboard script error on Windows

Describe the bug
Running npm run start-dashboard script produces an error on Windows when trying to run REACT_APP_TARGET=dashboard

Have you solved it? If so, let us know how!
Windows has to use the SET command to set environment variables. Below script works on my machine for starting the dashboard.

"start-dashboard": "SET \"REACT_APP_TARGET=dashboard\" && react-scripts start"

Thinking this command could be added to the package.json file as a separate script, something like "start-dashboard_win".


Screenshots and logs
Capture

🎨 Use colors from theme palette in NodePort.tsx

The file that should be updated is https://github.com/jina-ai/dashboard/blob/master/src/components/FlowChart/NodePort.tsx

In components like NodePort and NodePortInner, colors are hard-coded. Change this to use color from theme.

For example,
Change

  const NodePortInner = styled.div`
    background-color: #099;
    color: white;
    border-radius: 0.25rem;
    width: 0.5rem;
    height: 0.5rem; ;
  `;

to

  const NodePortInner = styled.div`
    background-color: ${(props) => props.theme.palette.primary};
    color: ${(props) => props.theme.palette.background};
    border-radius: 0.25rem;
    width: 0.5rem;
    height: 0.5rem; ;
  `;

πŸ§ͺ Create integration tests for LogsTable

πŸ§ͺ This is the view that should be tested

Screenshot from 2021-01-12 12-39-36

πŸ’» How to run this project locally

  1. Fork this repository by clicking on fork button on the top right. This will create a copy of this repository on your account.

  2. Clone your fork.
    Get the URL of your fork by clicking on code button on GitHub.
    Clone it to your local with command git clone <URL you copied>

    git clone <https://github.com/><your-username>/dashboard.git.

  3. Install dependencies using command npm i.

cd dashboard
npm i
  1. Run dashboard via the following ways .

Run in the debug mode

  1. npm run start_dev-server

    testServer will be running on http://localhost:5000 by default

  2. npm run start-dashboard

    Dashboard will be served on http://localhost:3000 by default

πŸƒβ€β™€οΈ How to get there.

Click on the option Log Stream from the sidebar on the left

The components of interest are LogsTable and LogsTableHeader

πŸ§ͺ How to run integration tests

Run npm run cy:dev from your console. This will open cypress and you can run the test suites you want to run by clicking on it.

When you're done working, make sure that npm run cy:run passes without any errors.

πŸ“ Where to make changes?

In cypress/integration/views directory and in the components you want to test.
Check this pull request

Pay attention to the changes in files cypress/integration/views/logsView.ts and src/components/LogStream/BarChartBase.tsx. You'll have to make similar changes.

πŸ“– How to learn about writing integration tests?

We use a framework called cypress for integration tests. Their documentation is quite helpful.
Go through their best practices, interacting with elements, structuring tests and you'll get a great understanding of how to write good integration tests

πŸ–οΈ I want to work on this.

If you're interested in working on this issue, please write a comment down below. πŸ’¬

If you need any help or have questions, you can ask questions here or ask in our slack group

the flow panel checks connectivity on wrong endpoint?

when stream endpoint is correct, flow yaml endpoint is wrong, clicking on flow panel will not show red bad connection alert. It seems that flow panel connectivity checks on the stream endpoint therefore does not complain?

pod api endpoint

jina-ai/jina#335

very first version of the pod api export, available at /data/api/pod (logserver.default.yml)

It returns {'pod': d, 'version': jina_version, 'usage': pod_cli_usage}, where d is the 'help', 'choices', 'default' and 'type' of each arg.

Limits:

  • enum is stringify to int and becomes 0,1,2... semantics are lost
  • default is only for reference, as some random default args will give random value everytime you call it.

🎨 Use colors from theme palette in CommandBar.tsx

The file that should be updated is https://github.com/jina-ai/dashboard/blob/master/src/components/FlowChart/CommandBar.tsx

In components like ButtonGroup and Button, background is hard-coded. Change this to use color from theme.

For example,
Change

const ButtonGroup = styled.div`
  height: 3.125rem;
  background: #009999;
  border-radius: 8px;
  display: flex;
`;

to

const ButtonGroup = styled.div`
  height: 3.125rem;
  background: ${(props) => props.theme.palette.primary};
  border-radius: 8px;
  display: flex;
`;

flow renders incorrectly for hello-world

!Flow
pods:
  chunk_seg:
    yaml_path: helloworld.crafter.yml
    replicas: $REPLICAS
    read_only: true
  doc_idx:
    yaml_path: helloworld.indexer.doc.yml
  encode:
    yaml_path: helloworld.encoder.yml
    needs: chunk_seg
    replicas: $REPLICAS
  chunk_idx:
    yaml_path: helloworld.indexer.chunk.yml
    replicas: $SHARDS
    separated_workspace: true
  join_all:
    yaml_path: _merge
    needs: [doc_idx, chunk_idx]
    read_only: true

gives a wrong rendering

image

the correct graph should be join_all receive outputs from doc_idx and chunk_idx.

Implement ImmutableJS for Redux store

Copy from https://redux.js.org/recipes/using-immutablejs-with-redux#why-should-i-use-an-immutable-focused-library-such-as-immutablejs

Immutable.JS was designed to provide immutability in a performant manner in an effort to overcome the limitations of immutability with JavaScript. Its principle advantages include:

Guaranteed immutability#
Data encapsulated in an Immutable.JS object is never mutated. A new copy is always returned. This contrasts with JavaScript, in which some operations do not mutate your data (e.g. some Array methods, including map, filter, concat, forEach, etc.), but some do (Array’s pop, push, splice, etc.).

Rich API#
Immutable.JS provides a rich set of immutable objects to encapsulate your data (e.g. Maps, Lists, Sets, Records, etc.), and an extensive set of methods to manipulate it, including methods to sort, filter, and group the data, reverse it, flatten it, and create subsets.

Performance#
Immutable.JS does a lot of work behind the scenes to optimize performance. This is the key to its power, as using immutable data structures can involve a lot of expensive copying. In particular, immutably manipulating large, complex data sets, such as a nested Redux state tree, can generate many intermediate copies of objects, which consume memory and slow down performance as the browser’s garbage collector fights to clean things up.

Immutable.JS avoids this by cleverly sharing data structures under the surface, minimizing the need to copy data. It also enables complex chains of operations to be carried out without creating unnecessary (and costly) cloned intermediate data that will quickly be thrown away.

You never see this, of course - the data you give to an Immutable.JS object is never mutated. Rather, it’s the intermediate data generated within Immutable.JS from a chained sequence of method calls that is free to be mutated. You therefore get all the benefits of immutable data structures with none (or very little) of the potential performance hits.

🎨 Refactor: existing styles to css-in-js with @emotion.

This comes from an action item in #127. We have made a decision to move to css-in-js to have atomic styles, to make it easy to use themes, make it easy to find and update styles.

The general idea is to the styles we currently have in css, scss files to css-in-js, use theming and improve, refactor our current emotion styles when needed

This issue will spawn multiple smaller issues. I'll keep track of them in a checklist below.

Change relative log streaming to absolute

The way Google Analytics handles log streaming is in an absolute time frame:

PoliticalFantasticBluebottle-small

But our logging time frame is relative:

Screen.Recording.2021-01-05.at.16.55.37.mov

I would suggest to change that to the way google handles it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.