middlewarehq / middleware Goto Github PK

Description:

cro :exclamation:️ tail: cannot open '/var/log/cron.log' for reading: No such file or directory

Steps to reproduce:

1. run `. ./dev.sh
2. On build completion you get above error

Expected behavior:

Actual behavior:

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Description:

just setup the project using ./dev.sh and the ui is breaking

Steps to reproduce:

1. Take a fresh pull
2. ./dev.sh

Expected behavior:

• Should load

Actual behavior:

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Description:

Added token but cant see any repos

Steps to reproduce:

1. Add token
2. Create team
3. Add repos

Expected behavior:

Load personal repos or give option for mocking data.

Actual behavior:

no repos loaded to add to team

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Description:

Steps to reproduce:

1. Go to dora-page
2. open team selector
3. edit a team
4. observe the team-selector

Expected behavior:

• reps number remains same

Actual behavior:

• should update team-name/repo-count

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Why do we need this ?

The Team Repos and Team Incidents Logic should have not been added to Incident Service, Refactor it to it's own service

Acceptance Criteria

• Team Incidents Service is updated while updating Team Repos, the code needs refactoring.

Further Comments / References

Description:

Steps to reproduce:

1. Go to integration page
2. Add PAT for github
3. Click on create team

Expected behavior:

• Team management UI should open

Actual behavior:

• Page reloads and nothing happens

Server Setup Information:

Using Docker ARM build.

• Operating System: Using Docker ARM build on Mac OS 14 on M2.

Additional context

Description:

• There is an error log ion the console for sync breaking to null datetime felid being converted to iso string.

Steps to reproduce:

1. Add Integration PAT
2. View Logs

Expected behavior:

• Data sync should be without error logs

Actual behavior:

• Error logs in data sync.

Server Setup Information:

• Operating System: Using Docker ARM build on Mac OS 14 on M2

Additional context

Why do we need this ?

We need a way to get the devs to quick start with development using Gitpod. This would enable devs who dont have machines capable of supporting the setup.

Acceptance Criteria

• Add a gitpod yml file that builds the project as soon as we open this on Gitpod
• Add a Link to open the repo on Gitpod

Further Comments / References

Example: https://github.com/RocketChat/Rocket.Chat?tab=readme-ov-file#gitpod-setup

Why do we need this ?

There are cases when a new column is added which is populated at the time of the sync. This change will not reflect in db existing data, but will be there in data synced after the version upgrade.

Example: We add a new url column in Incident Table like we did in this PR #245 . This uses some logic to populate url column in the sync itself but the existing rows will have null url. We should have a mechanism to do such database migrations as the logic can be more complex than a db query and re-syncing all data can be a pain.

Acceptance Criteria

• Build a service/system that can track data migrations that need to be performed on the existing data.
• It should be easy to create and setup the migrations for the developer.
• User should be able to run these migrations automatically or via some scripts on any version update.

Further Comments / References

Description:

when fetching deployment PRs, API fails

Steps to reproduce:

1. go to dora metrics
2. open deployment frequency
3. click on a deployment card

Expected behavior:

• should load up the prs for the deployment

Actual behavior:

• API fails

Additional context

Logs:

TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'
[2024-05-03 09:33:12,310] ERROR in app: Exception on /deployments/PR_MERGE|1054f54b-d19d-4ec2-8d6b-c2faeb633c30/prs [GET]
Traceback (most recent call last):
  File "/opt/venv/lib/python3.9/site-packages/flask/app.py", line 1463, in wsgi_app
    response = self.full_dispatch_request()
  File "/opt/venv/lib/python3.9/site-packages/flask/app.py", line 872, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/opt/venv/lib/python3.9/site-packages/flask/app.py", line 870, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/venv/lib/python3.9/site-packages/flask/app.py", line 855, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
  File "/app/backend/analytics_server/mhq/api/deployment_analytics.py", line 135, in get_prs_included_in_deployment
    return get_non_paginated_pr_response(
  File "/app/backend/analytics_server/mhq/api/resources/code_resouces.py", line 70, in get_non_paginated_pr_response
    "data": [
  File "/app/backend/analytics_server/mhq/api/resources/code_resouces.py", line 82, in <listcomp>
    "lead_time": _get_lead_time_for_pr(pr),
  File "/app/backend/analytics_server/mhq/api/resources/code_resouces.py", line 10, in _get_lead_time_for_pr
    (
TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'
[2024-05-03 09:33:12,320] ERROR in app: Exception on /deployments/PR_MERGE|1054f54b-d19d-4ec2-8d6b-c2faeb633c30/prs [GET]
Traceback (most recent call last):
  File "/opt/venv/lib/python3.9/site-packages/flask/app.py", line 1463, in wsgi_app
    response = self.full_dispatch_request()
  File "/opt/venv/lib/python3.9/site-packages/flask/app.py", line 872, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/opt/venv/lib/python3.9/site-packages/flask/app.py", line 870, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/venv/lib/python3.9/site-packages/flask/app.py", line 855, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
  File "/app/backend/analytics_server/mhq/api/deployment_analytics.py", line 135, in get_prs_included_in_deployment
    return get_non_paginated_pr_response(
  File "/app/backend/analytics_server/mhq/api/resources/code_resouces.py", line 70, in get_non_paginated_pr_response
    "data": [
  File "/app/backend/analytics_server/mhq/api/resources/code_resouces.py", line 82, in <listcomp>
    "lead_time": _get_lead_time_for_pr(pr),
  File "/app/backend/analytics_server/mhq/api/resources/code_resouces.py", line 10, in _get_lead_time_for_pr
    (
TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'
^C

Screenshots

Description:

Steps to reproduce:

1. Go to /dora-metrics
2. add a custom branch name ^main
3. refresh this page
4. open branch-selector again
5. branch-chips disappear and you cannot unselect particular branch names

Expected behavior:

• branch names should render as removable-chips

Actual behavior:

• no rendered values that can be de-selected

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Description:

Steps to reproduce:

1. Create a team-A and a team-B
2. edit one of them
3. copy the modal-url
4. close the url
5. delete team
6. open the modal-url
7. UI breaks

Expected behavior:

• handle breakage

Actual behavior:

• UI breaks

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Description:

The code base is divided into two portions, the web-server (typescript/next js) and apiserver (python/flask). pytests and black workflows should only run when changes are made to python directory or apiserver as they are focused towards that.

At the moment these workflows run for all PRs and may be an overhead for devs contributing strictly to web-server.

Steps to reproduce:

1. Make Changes to web-server
2. Check Workflow runs

Example: #223

Expected behavior:

• Python and flask centric workflows should run only when changes are made to apisever.

Actual behavior:

• pytest and black run on all PRs

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Description:

• Team selector shows 0 repos on DORA metrics even though repos exist

Steps to reproduce:

1. Refresh On DORA metrics page
2. Select team with some repos

Expected behavior:

• Team selector shows repos along side team name

Actual behavior:

• team repo count is zero
  

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Description:

When I am integrating the project and selecting repos for the team, I am unable to see all my repos.

This is blocked because of the pagination issue.

Expected behavior

We should be able to navigate through all the pages of repos available.

Why do we need this ?

Currently, the _set_unused_repos_as_inactive method defies the single responsibility, that a method should have.
Decouple it and create APIs for each small part.

Acceptance Criteria

• Decoupled logical methods
• Decoupled APIs
• BFF updated to use all the decoupled APIs.

Description:

Steps to reproduce:

1. Try changing branches on dora-metrics page
2. see the data being passed as payload to the API in network call

Expected behavior:

should see branches as a key in payload

Actual behavior:

no branches in payload

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Description:

• When we delete the container and rebuild it, all the data is wiped out
• But singleTeam stays persisted
• This causes a wrong team to be selected on first render

Steps to reproduce:

1. select a team, and note down it's team-id
2. delete the docker-container
3. start again with the ./dev.sh setup
4. link token, create teams, and finally go to the dora-metrics page.
5. note the team-id of the selected singleTeam
6. dora_metrics API fails

Expected behavior:

• single team from the new fetched teams should be selected

Actual behavior:

• older persisting teams are being selected as singleTeam by default

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

dora_metrics API gives 404

Description:

• Editing a team goes into the loading state forever, only reloading fixes it.

Steps to reproduce:

1. Go to /teams page
2. create a team or select a pre-existing one to edit
3. Edit org-repos
4. click save

Expected behavior:

• should save, and refresh all teams

Actual behavior:

• keeps getting stuck at the loading state

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Why do we need this ?

id column that is primary key in TeamIncidentService is not required, we can drop it and use (team_id, service_id) as primary key.

Acceptance Criteria

• DROP id column from TeamIncidentService, make (team_id, service_id) as primary key
• Check usages of TeamIncidentService.id in codebase and remove them

Further Comments / References

Description:

• The integration screen has a url to generate token but that links to the Github Doc instead of token generation page.

Steps to reproduce:

1. Go to integrations
2. Click on Link Integration
3. click on the link below.

Expected behavior:

1. The link shoul take me to https://github.com/settings/tokens to generate my classic token.
2. It should be mentioned that the user should generate a classic token.
3. Github PAT doc link is not needed.

Actual behavior:

Server Setup Information:

Using Docker ARM build.

• Operating System: Using Docker ARM build on Mac OS 14 on M2.

Additional context

Description:

The ARM build for docker images is taking an excessively long time (around 1 hour) and failing, while the AMD build is successful and takes only 5 minutes.

Steps to reproduce:

1. Trigger the workflow to build the Docker images.
2. Observe the build process and timings for both ARM and AMD platforms.

Expected behavior:

Both ARM and AMD builds should complete successfully within a reasonable time frame, without any significant difference in build times between the two platforms.

Actual behavior:

The ARM build takes around 1 hour and fails, while the AMD build completes successfully in just 5 minutes.

Server Setup Information:

• Operating System: Ubuntu (GitHub Actions runner)
• Deployment Method: Docker

Additional context

This issue seems to be related to the performance and compatibility of the ARM build process with the GitHub Actions runner environment. Additional investigation is needed to identify the root cause of the excessive build time and failure for the ARM platform. Potential factors to consider include:

• Compatibility of the ARM build environment with the specific dependencies or tools required for the project.
• Optimization of the build process for ARM architectures.
• Resource allocation and constraints within the GitHub Actions runner environment for ARM builds.

It would be beneficial to review the build logs and error messages for the failed ARM builds to gain more insights into the specific issues encountered.

Description:

Pressing enter while pasting the linked token on the link-integration token closes the modal and does nothing

Steps to reproduce:

1. Go to /integrations
2. click "link"
3. add token to the text-field
4. press enter

Expected behavior:

• should submit the token and link

Actual behavior:

• closes and does nothing

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Why do we need this ?

TeamRepos.prod_branch is no longer used in analytics, rather TeamRepos.prod_branches is used.

Acceptance Criteria

• Find it's usages and deprecate them
• Drop it from DB migration script

Why do we need this ?

As we have kept interactive CLI for developer mode on hold hence, add command on how to access the logs of individual applications.

Acceptance Criteria

• Readme should contain commands to access all the necessary logs
• All the commands should be system agnostic or commands should be added for all windows, macOS and linux.

Why do we need this ?

• Our readme is missing the manual setup to run web-server

Acceptance Criteria

• Add manuals setup for web-server

Further Comments / References

Description:

Running ./dev.sh shows that apiserver is loading a seeminly infinite amount of logs that causes the container storage to fill up and crash other things

Steps to reproduce:

1. Run ./dev.sh
2. Wait for dockerfile setup steps to run though
3. Eventually see logs from api, web, and other sources to start streaming in
4. Notice that api logs keep coming in rapidly, and of course since this is being written to the disk, it takes up all the space

Expected behavior:

It should not create so many logs as to fill up the disk.

Why is this required?

The lead time attribute in the PullRequests model is no longer cached and should be deprecated. This change was made to align with the new calculation method for merge to deploy, which now depends on the deployment type for TeamRepos being either PR_MERGE or WORKFLOW.

Scope:

1. Remove lead_time attribute usage from frontend system for pull requests.

   The frontend system should no longer rely on the lead_time attribute when displaying pull request data.

2. Remove lead_time attribute usage from core backend APIs that return Pull requests data.

   Update core backend APIs to no longer return the lead_time attribute when fetching pull request data.

3. Drop lead_time column from the database.

   Remove the lead_time column from the database schema as it is no longer required.

Why do we need this ?

• It would be good to see Sync Server Logs to see progress of sync in the cli when you do ./dev.sh

Acceptance Criteria

• Track Logs of flask sync server in the terminal UI.
• Add ability to switch between sync and other logs.

Further Comments / References

Description:

Steps to reproduce:

On the dora metrics page:

1. Select a team (already selected by default)
2. Open its deployment frequency
3. now close the overlay
4. select another team
5. open the deployment frequency overlay again

Expected behavior:

First repo should be auto-selected

Actual behavior:

No-repos are selected, also we get this:

Additional context

Description:

I was unable to run dev.sh as requested because of a package version mismatch.

Can we please mention the pre-req versions for each project dependency?

Why do we need this ?

• The current PRs have no structure.
• We need a PR pattern that is followed for all prs of middleware

Acceptance Criteria

• Add Easy to use and extensible PR template in .github

Further Comments / References

Description:

• if you click on change-failure card before lead time, and then open lead-time card, lead-time becomes empty

Steps to reproduce:

1. Go to /dora-metrics
2. open lead-time overlay by clicking on the card
3. observe it is not empty
4. close overlay
5. open change-failure rate overlay by clicking the card
6. close the overlay
7. open lead time overlay again

Expected behavior:

• data should remain same

Actual behavior:

• data empties out

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Description:

Cant see webserver logs for docker

Steps to reproduce:

1. dev.sh
2. docker exec -it middleware-dev tail -f /var/log/web-server/web-server.log

Expected behavior:

1. Log trail

Actual behavior:

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Description:

• On visiting the teams page, even when teams are present, a fetch call is made
• Again when opening edit-overlay, fetch request is made
• Again when closing the teams page, fetch request is made

Steps to reproduce:

1. Go to teams page after integrations page
2. Open edit overlay
3. close edit overlay

Expected behavior:

1. Teams page should only fetch data if no data is present, or if data is updated (CRUD operations)

Actual behavior:

1. Too many reloads

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Why do we need this ?

• A team may be interested in seeing how many PRs got merged without review in a time duration.
• This can help in setting processes within the team

Acceptance Criteria

• Add Python Backend API to fetch Teams PRs merged without review for a team.
• Update Next JS BFF slices and state to fetch and persist a team PRs merged without review.
• Add UI component to see PRs merged without review for a team.

Task 1, 2, 3 can be done in seperate PRs.

Further Comments / References

Description:

Repo names are clipped when I search then in the integrations page

Expected behavior:

I should be able to see full names

Actual behavior:

When I search my repos, the full names get clipped, preventing me from understanding the right one.

Description:

Our logo and brand colors seem to be different when I see it in light theme and dark theme.

The dark theme seems to be closer to our brand colors.

Light Theme

Dark Theme

Ideal state

Colors are uniform across modes.

Why do we need this ?

• We need to find if there are any bugs in. the MTTR calculation.

Acceptance Criteria

• Add Unit tests for APIs calculating mean time to recovery.
• Fix any bugs along the way

Further Comments / References

Description:

We currently have a single log stream for server and analytic server.

Steps to reproduce:

1. ./dev.sh
2. Wait for container to start
3. Switch to api logs

Expected behavior:

• Separate log streams for analytics sever

Actual behavior:

• Single screen for sync and analytics

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Description:

• Editing teams shows previously selected repositories as unselected, allowing users to re-choose them. Ideally, the list should automatically reflect the team's repos

Steps to reproduce:

1. Go to any page that supports team-edit (eg: /teams)
2. Open the team-edit overlay
3. search for a selected repo

Expected behavior:

• search result should reflect that the previously selected repos are already selected
• should deselected that repo when clicked

Actual behavior:

• search results don't reflect this selected
• allows re-selection of repos
• deleting repeated selections removes all same selections

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

• Possibly change teamReposMaps from Record<string, DB_OrgRepo[]> to Record<string, BaseRepo[]>

Description:

the labels org-name/repo-name is a frontend generated logic. hence searching for org-name/repo-name yields no results. Currently only repo-names are used to search and fetch repos. it should also allow searching on the organization

Steps to reproduce:

1. Go to teams page
2. create team by clicking "Add team"
3. search for repos
4. search for org-name/repos

Expected behavior:

1. should filter out org-names

Actual behavior:

2. no logic on the org-name

Additional context

Why do we need this ?

• Overriding the search query in the search-repo should cancel the previous request.

Acceptance Criteria

Further Comments / References

Why do we need this ?

Currently a lot of logic for team repos is written in Repo Layer.
The logic should remain in service layer.

Refer #147 for changes required

Acceptance Criteria

• Move Logical Code to it's own service layer

Further Comments / References

#147

Description:

Currently we create encryption secret keys on runtime, but if container is brought down and volumes persists then the next the container is spun up, it would lead to encryption decryption enconsistencies

Steps to reproduce:

1. Run the docker container with volume persistence
2. Use the app to populate the DB with integration token
3. Stop the container and restart the container.

Expected behavior:

The keys should also persist and not be created if they exists

Actual behavior:

New keys are created and leads to wrong decryption

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Description:

When we use the dev CLI setup with ./dev.sh which internally uses docker compose watch, the database is not persisted once we stop the setup container.

Steps to reproduce:

1. ./dev.sh
2. Let the data sync
3. Check your DB data in a db client like table plus. The db configs can be taken from env.example
   
4. Close the cli app or container
5. Restart the CLI ./dev.sh
6. Check Database

Expected behavior:

• Synced data should be persisted

Actual behavior:

• Data is deleted and is re-synced

Server Setup Information:

• Operating System:
• Deployment Method:

Additional context

Description:

A user might be a part of multiple orgs and while selecting the repositories, it becomes confusing to find the right org.

Expected behavior:

It would be great to choose the orgs first. This will also help to reduce the clutter in the Add Repos option.

Actual behavior:

Today, it loads up a list of repos across my personal and from other orgs.

Description:

Whenever I run the product in dev mode, I am not able to switch logs between web server and api server.

Steps to reproduce:

1. ./dev.sh
2. switch between server logs. and switch to api-server logs. [w]

Expected behaviour:

• It should switch logs to API server.

Actual behavior:

• Cant switch between logs.
  

Server Setup Information:

• Operating System: Mac OS 14
• Deployment Method:

Additional context

Can we also mention the system pre-requisites like yarn and python to be there to run this dev.sh script?