newrelic / nr1-catalog Goto Github PK

New Relic One Catalog repo of repos

License: Apache License 2.0

JavaScript 100.00%

nr1-catalog's Introduction

nr1-catalog

This project is very much in development and not ready for public contribution. It's just being developed in the open.

New Relic One Catalog repo of repos. PR's to this repo trigger the review process for either adding a New Relic One project via the NR1 CLI or upgrading an existing NR1 project.

Usage

This repository is meant to be a repository of submodules that each represent a codebase designed to fit into the New Relic One Catalog. The repository contains three main components:

A collection of submodules each containing an New Relic One application.
GitHub Action automation to handle the verification of PR's and promotion of submodules on a merge to master.
Documentation on the standard(s) for New Relic One Catalog projects.

That functionality manifests in three distinct use cases.

Use Case 1: PR with a new submodule

Actors: Application Developer(s), New Relic Open Source Maintainers

To submit a new New Relic One project (application, etc.) for consideration and review:

Make sure your project's repo is public
Fork this repo
Add your submodule using the git submodule command

cd nr1-catalog/apps
git submodule add [YOUR REPO URL]
# example: git submodule add https://github.com/newrelic/nr1-browser-analyzer.git
git commit -a -m "feat: Adding [YOUR REPO NAME] to nr1-catalog."

Submit a PR to this project

From there, New Relic Open Source Maintainers will validate (using a combination of tooling and manual review):

Project structure (does it comply with the technical requirements)
Compliance with architectural and design standards
Compliance with security standards
Validation of the documentation and support standards
Assignment of a uuid under the global catalog account to this repo (this will need to be generated by someone with access to the account)

Once your PR has been approved and merged it will publish your app to Production.

The catalog maintainers will then update your nerdpack's subscription model to GLOBAL*

*for maintainers: see the hero FAQ doc

To have your app available in the Staging and EU environments: Reach out to the platform services team with your app name, UUID (found in this repo's globals.json file), and desired version so they can tag it in Staging and EU.

Use Case 2: PR to an existing submodule

Actors: Application Developer(s), New Relic Open Source Maintainers

This step verifies the same requirements as Use Case 1, but the review process is focused on the difference between versions.

Use Case 3: Merge of a PR to the master Catalog

Actor: New Relic Open Source Maintainers

When a New Relic Open Source Maintainer merges a PR to master, the referenced submodule is not only checked out, built, and associated to it's global uuid; but the following also occurs:

the New Relic One project is published to the global catalog account,
the New Relic One project is deployed to the STABLE channel of the global catalog account,
and the catalog registry is updated with the latest meta data

Use Case 4: Removal of an Application from the NR1 Catalog

Actor: Application Developer(s), New Relic Open Source Maintainers

Please refer to this WIKI for details on this process.

nr1-catalog's People

Contributors

Stargazers

Watchers

nr1-catalog's Issues

Tracking: Snyk usage for PRs

Tracking issue - identify a way to use Snyk with PRs that come from outside contributors to validate security of code.

Create / config an Action that validates that a submodule contains the Catalog components

The specs for the catalog are described in detail here and usage in the proposed UI is here.

We need:

a semantic version number in the package.json
an nr1.json file in the root of the submodule
an icon.png file in the root of the submodule
a catalog/config.json file matching the linked format (containing at least two elements in the support subsection of the JSON)
at least one screenshot in the catalog/screenshots directory

Update nr1-pageview-map

Acceptance Criteria

Upgrade to node 16
If applicable resolve any app deprecations planned for march 1st.
- You are not implementing property "NerdletSchema.actionCategory" as specified in the docs
- You are implementing a deprecated property "NerdletSchema.entities", please use "NerdletSchema.context.entities" instead
- The "accountId" prop is deprecated, use the "accountIds" prop instead.
- You are not implementing property "NerdletSchema.context.entities" as specified in the docs
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

Update nr1-learn-nrql

Acceptance Criteria

Upgrade to node 16
If applicable resolve any app deprecations planned for march 1st.
- The "accountId" prop is deprecated, use the "accountIds" prop instead.
- "normal" type for header controls has been deprecated. Use any of "primary", "secondary", "tertiary" instead
- "plain" type for header controls has been deprecated. Use any of "primary", "secondary", "tertiary" instead
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

Update nr1-neon

Acceptance Criteria

Upgrade to node 16
If applicable resolve any app deprecations planned for march 1st.
- "plain" type for header controls has been deprecated. Use any of "primary", "secondary", "tertiary" instead
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

Update nr1-deployment-analyzer

Acceptance Criteria

Upgrade to node 16
If applicable resolve any app deprecations planned for march 1st.
- "accountId" is deprecated, use "accountIds" instead
- The "accountId" prop is deprecated, use the "accountIds" prop instead.
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

Update nr1-groundskeeper

Acceptance Criteria

Upgrade to node 16
If applicable resolve any app deprecations planned for march 1st.
- "errors" is deprecated, use "error?.graphQLErrors" instead
- "errors" is deprecated, use "error" instead
- "plain" type for header controls has been deprecated. Use any of "primary", "secondary", "tertiary" instead
- "normal" type for header controls has been deprecated. Use any of "primary", "secondary", "tertiary" instead
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

Update nr1-browser-analyzer

Acceptance Criteria

Upgrade to node 16
Resolve any app deprecations planned for march 1st.
- "accountId" is deprecated, use "accountIds" instead
- ~~You are not implementing property "NerdletSchema.actionCategory" as specified in the docs~~
- ~~You are implementing a deprecated property "NerdletSchema.entities", please use "NerdletSchema.context.entities" instead~~
- relationships: This field is deprecated! Please use relatedEntities instead.
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

newrelic/nr1-browser-analyzer#85

Platform Deprecations] The following apps require updates

Description

The following apps and deprecation messages have been logged in the platform. To prevent users of these apps seeing error toasts in the platform, please make the necessary changes.
See for more context:

Once the PR's are merged we will need to update each app in the catalog to the latest version

Apps

Update nr1-observability-maps

Acceptance Criteria

Upgrade to node 16
If applicable resolve any app deprecations planned for march 1st.
- The "accountId" prop is deprecated, use the "accountIds" prop instead.
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

[nr1-graphiql-notebook] Build Error - TypeError: Cannot set property 'chunkName' of undefined

Description

This seems to be coming from the nr1-cli when trying to build the app both in GH and locally. Here is the PR build for context

https://github.com/newrelic/nr1-graphiql-notebook/pull/59/checks?check_run_id=3771747438

I've reached out to the team responsible for the CLI to investigate this so will update

AC

Error is fixed and app builds successfully
#288 should be resolved and building correctly

Automate the internal Slack message on successful master merge

@[name] can we please get [nerdpack_name] ([global uuid]) advanced to [version] for [environments array]? Thank you.

Remove the 3rd party JFR nerdlet from staging and archive the repo

Currently, in the staging environment there are two JFR nerdlets available, the old third party nerdlet and the first party nerdlet that replaced it.

We should remove the third party nerdlet from staging so that only the first party nerdlet is visible. We should also archive the nr1-jfr repo since it's been replaced.

@amit-y mentioned the following blocker which needs to be addressed before we can remove the nerdlet:

we need to identify the accounts which have access to the older version, and then unsubscribe them. I have reached out to get a list of accounts that are currently subscribed through the backend, because there's no way to get that through the UI. I haven't heard anything back from that team. So, essentially we'd have to remember all the accounts that we enabled, and then go in and unsubscribe all of them.

Create GitHub Action(s) to merge a PR of an NR1 Application

On merging a Pull Request to master, perform the following:

(if possible) Identify the submodule referenced in the PR (assuming one submodule change per PR)
Checkout / update the submodule
Build the application
Lint the code This was taken care of during PR.
Lookup a uuid in a repo-traceable local data store that maps this submodule to the global catalog account. If none exists, generate one and save the uuid. If it exists, use that value to perform the next step. See #6
Publish the code #2
Publish the catalog data #1
(if possible) Notify the appropriate list(s) via Slack of the published change

Create / config an Action that builds the NR1 package validates the ability to publish it to NR1

Execute nr1 nerdpack:publish --dry-run on the PR'd code in the submodule

Define our approach to support our Open Source repos during week of `aug 9 - aug 13`

🏖️ 🌞 🥳 🍹

Acceptance Criteria

see Austin's internal post and pinned channel message for guidance.
draft our approach & review with team
share internally
pin a message in each of our hero channels.

[Platform Deprecations] Scope out the changes needed for 3rd-party-apps

Description

Schema properties are being deprecated.
Users of affected nerdlets should be notified 12 months in advance via warnings logged in the console of applications. Then 1 month before deprecation red error toasts are posted in the platform UI when the nerdlet is loaded.

This posts these toast messages for customers as well so there have been support tickets around them, as they've caused some confusion.

We will need to scope out the work needed on our end to alter or remove the affected properties. These are generally located in the nr1.json schema of a nerdlet.

Some examples are actionCategory being removed and entities changing to context.entities.

Example PR with changes:

newrelic/nr1-event-stream#59

Acceptance Criteria

The work to implement the needed changes is scoped and ticketed
Assess what changes need to be made to our workflow so we can address these in good time

Nr1 App Catalog Team Technical Deep Dive

In order to better prepare for the Quick Starts initiative we'll walk through the technical details of the NR1 App Catalog as we'll be doing work in there soon.

Acceptance Criteria

@zstix and @LizBaker lead the team through a deep dive
include Ruairí if possible. 🇮🇪
Determine how we can work with Developer Experience team to add into the catalog

Create better automation for nr1 apps

Create a GitHub action that can be added to NR1 app repos which, on a commit to the release branch (master or main):

checks out a branch of THIS repo
updates that branch's submodule to the latest release hash of the given NR1 app
submits a PR for a specific commit of that branch to THIS repo's release branch (currently master)

The objective is to automate the release process for permissioned apps (especially NR-owned apps)

Update nr1-nimbus

Acceptance Criteria

Upgrade to node 16
If applicable resolve any app deprecations planned for march 1st.
- data.chart is deprecated, use data instead
- "accountId" is deprecated, use "accountIds" instead
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

Update nr1-event-stream

Acceptance Criteria

Upgrade to node 16
~~If applicable resolve any app deprecations planned for march 1st.~~
- ~~You are not implementing property "NerdletSchema.actionCategory" as specified in the docs~~
- ~~You are implementing a deprecated property "NerdletSchema.entities", please use "NerdletSchema.context.entities" instead~~
~~update .nvrmc file to Node 16~~
~~Confirm if we need to set the NODE_VERSION environment variable~~
~~Ensure all documentation is updated.~~

Add react pinned library checks to PR and merge CI

Docs highlight pinned react and react-dom libs to 16.6.3. We need to automate that in the PR and merge CI

Update nr1-pathpoint

Acceptance Criteria

Upgrade to node 16
If applicable resolve any app deprecations planned for march 1st.
- "errors" is deprecated, use "error?.graphQLErrors" instead
- "errors" is deprecated, use "error" instead
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

Tracking: Passing secrets to pull_request events in Actions (future state)

Opening this issue to keep track of possible future state where working with secrets in Actions is a bit (lot) more robust.

Currently:

With the exception of GITHUB_TOKEN, secrets are not passed to the runner when a workflow is triggered from a forked repository.

https://help.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets#using-encrypted-secrets-in-a-workflow

Relevant community threads:

Create / config an Action that validates the expected structure of an NR1 application

There are a number of commands and configurations that we're recommending / enforcing internally w/r/t to NR1 apps w/in New Relic. We want to be sure we're validating that these configurations exist so that we have the capability to execute things like linting commands further downstream in the process.

In addition, the package.json should contain the following commands:

scripts.eslint-check
scripts.eslint-fix

See our current, standard circleci config as a pattern.

Create GitHub Action(s) to process a PR of an NR1 Application

On the initiation of a Pull Request:

(if possible) Identify the submodule referenced in the PR (assuming one submodule change per PR)
Checkout / update the submodule
Build the application
Lint-check the code
Validate the structure as an application #3
N/A - See Comment Lookup a uuid in a repo-traceable local data store that maps this submodule to the global catalog account. If none exists, generate one and save the uuid. If it exists, use that value to perform the next step. See #6.
N/A - See Comment ~~Validate the ability to publish the code #2~~
Validate the ability to publish the catalog data #1
Execute a security and licensing check using Synk

Update nr1-kentik-network-monitoring

Acceptance Criteria

Upgrade to node 16
If applicable resolve any app deprecations planned for march 1st.
none found at time of making the ticket, be sure to check the terminal and browser console when running locally
~~update .nvrmc file to Node 16~~
~~Confirm if we need to set the NODE_VERSION environment variable~~
~~Ensure all documentation is updated.~~

[Platform Deprecations] The following apps require updates

Description

Once the PR's are merged we will need to update each app in the catalog to the latest version

Apps

Include a script to allow developers submitting apps to pre-validate their PR

Summary

From @Kav91:
Could we have a “catalog-ready” script added, like a cut down version off validate-catalog-requirements? So say before someone tries to add the submodule, in the steps you tell them to run the script to ensure it has everything in the checklist. Then anything missed can be linked back to the docs as the message.

I think it could save some time in the PR process.

Update nr1-cloud-optimize

Acceptance Criteria

Upgrade to node 16
Resolve any app deprecations planned for march 1st.
- You are not implementing property "NerdletSchema.actionCategory" as specified in the docs
- You are implementing a deprecated property "NerdletSchema.entities", please use "NerdletSchema.context.entities" instead
- data.chart is deprecated, use data instead
- The "accountId" prop is deprecated, use the "accountIds" prop instead.
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

3rd Party App Catalog Support Analysis

Problem

Can we get more data to understand the commitment to keeping lights vs enhancing and supporting these apps? The idea would be able to provide some data and estimates of effort to do more meaningful work with the app catalog. In doing this analysis, it would be good to decide which apps should still be available in the app catalog.

Acceptance Criteria

Add aggregate charts for 3rd Party Catalog EOL Dashboard to understand the usage and github activity
Look at enhancements and bugs and get sense of total effort and commitment needed to execute on these backlogs.
Provide a recommendation on how to proceed.

`semantic-release` breaking changes

Summary

The semantic-release NPM module has been updated to v18.0.0 with a breaking change that requires Node.js v14 or higher.
Semantic release v18

Most of our third party apps use this module to handle creating new releases, but do not lock down the module's version. So these workflows will now be attempting to use v18 of semantic-release.

Acceptance Criteria

To resolve this problem we need to update the Node.js version in these workflows for all the repos below to use >14.17 and lock down the semantic-release version to 18.0.0.

Affected repos

Using Terraform to manage all our repos

We have a lot of repos and need a lot of similar things, like labels, milestones, and permissions to things, can we leverage terraform to do some of that for us?

https://registry.terraform.io/providers/integrations/github/latest/docs/resources/issue_label

https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository

https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_milestone

Create a repo-based data store for mapping account id's to submodule names

We need the ability to lookup and save a mapping between an account id (by default the global account ID) and a submodule name of a uuid.

Constraint: this obviously implies that this project requires submodule names to be unique. This appears to be an acceptable risk for this phase of the process.

EOL nr1-graphiql-notebook

Acceptance Criteria

Confirm with the programmability 🧙 that there are no concerns with removing this app.
Follow the end of life process to remove this application from the catalog
Archive the repo

Usage

Improve submodule command info in the docs site

Some things to possibly add to documentation based on experiences with PRs so far.

Add example commands for adding and updating submodule to the add-checklist and update-checklist pages. They're present in the process outline on the home page, but might be helpful to have them quickly reference-able if contributors skip directly to the Add Nerdpack/Update Nerdpack page.
Add submodule section to Reference Docs
- Possibly detail how to update submodule in the same PR to avoid contributors feeling like they need to close and open a new PR when changes are requested
Add detail that submodules URLs should be HTTPS rather than SSH on the Add Nerdpack checklist and Reference Docs
(Possibly) Add detail about automation to Reference Docs
- Detail checks for the PR, including how it also runs when a PR is updated with new commits
- Deploy process

Merge all SYNK PRS for 3rd Party Apps

We have lot SNYK PRs that are queuing up in our app repos so we should merge those in and close the invalid ones.

Acceptance Criteria

Visit the App Catalog GH project board
Review the In Review column
Close all Snyk PRs that are labeled invalid as they are old and have failed a lot of checks.
merge in all SNYK PRs that pass checks
ensure any merge conflicts with package.json or lock files are resolved.
If any new SNYK PRs fail, add a invalid label and we'll evaluate next steps.

[3rd Party Apps] 💽 Node Upgrades + App Deprecations

EOL for Node 12 is apr 2022

Epic Milestones

upgrade all apps to node 16
Resolve all app deprecations
Add deprecations to repolinter deprecation rule set
Define code owners for these repos
Prepare for the handoff of 3rd party apps to the new team

Epic Plan

https://docs.google.com/document/d/1wvckxtOzZKbUg-PLal-ph_sbnjO2295x2AyFUcRH9V4/edit?usp=sharing

Deprecation Dashboard for validation

https://staging.onenr.io/0xVwglvklRJ

Epic project board

This epic will be managed using this project board: https://github.com/orgs/newrelic/projects/44

Questions

Determine if we can just go to node 16?
#304

Apps

Failure of 2 apps to update in the Catalog via the Catalog Manager

the following apps can't be updated in the catalog using the catalog manager:

nr1-datalyzer

0.6.1	v0.7.1

112	750050643	completed	failure	2492447	2021-04-14T23:04:40Z	https://github.com/newrelic/nr1-catalog/actions/runs/750050643

nr1-event-stream

1.4.0	v1.4.2

111	750048649	completed	failure	2492447	2021-04-14T23:03:58Z	https://github.com/newrelic/nr1-catalog/actions/runs/750048649

Create a repo-based data store for mapping account id's to submodule names

We need the ability to lookup and save a mapping between an account id (by default the global account ID) and a submodule name of a uuid.

Constraint: this obviously implies that this project requires submodule names to be unique. This appears to be an acceptable risk for this phase of the process.

Update nr1-account-maturity

Acceptance Criteria

Upgrade to node 16
Resolve any app deprecations planned for march 1st
- "errors" is deprecated, use "error?.graphQLErrors" instead
- "errors" is deprecated, use "error" instead
- "plain" type for header controls has been deprecated. Use any of "primary", "secondary", "tertiary" instead
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

Update readme with details for adding a new global app

During the latest app addition we realized the information about how to get an app added is pretty spread out.
This ticket tracks updating the readme in this repo with a more detailed set of steps.

Update git submodule command
Add steps after it's merged into the catalog (how to get the global tag added, etc)

Update nr1-datalyzer

Acceptance Criteria

Upgrade to node 16
Resolve any app deprecations planned for march 1st.
- ~~You are not implementing property "NerdletSchema.actionCategory" as specified in the docs~~
- ~~You are implementing a deprecated property "NerdletSchema.entities", please use "NerdletSchema.context.entities" instead~~
- The "accountId" prop is deprecated, use the "accountIds" prop instead.
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

newrelic/nr1-datalyzer#66

Confirm with Platform services that we support node 14 or node 16?

Acceptance Criteria

Answer this question

as part of the some future EOL planning I am doing for our 3rd party apps ecosystem, we are planning on upgrading all 3rd party apps to Node 16 in March 2022 since Node 12 is EOL as of April 2022. Are there any reasons why we should not upgrade to Node 16?

From the NR1-Tools team: There should not be any problem upgrading, but we should try first with a couple of internal nerdpacks

Update nr1-container-explorer

Acceptance Criteria

Upgrade to node 16
Resolve any app deprecations planned for march 1st.
- You are not implementing property "NerdletSchema.actionCategory" as specified in the docs
- You are implementing a deprecated property "NerdletSchema.entities", please use "NerdletSchema.context.entities" instead
- "errors" is deprecated, use "error?.graphQLErrors" instead
- "accountId" is deprecated, use "accountIds" instead
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

Update nr1-integrations-manager

Acceptance Criteria

Upgrade to node 16
If applicable resolve any app deprecations planned for march 1st.
- "errors" is deprecated, use "error" instead
~~update .nvrmc file to Node 16~~
~~Confirm if we need to set the NODE_VERSION environment variable~~
~~Ensure all documentation is updated.~~

Evaluate approach to refactor NR1 Github Nerdpack to reduce security bugs

The NR1 Github Nerdpack has several design flaws which have lead to 3 security p1 bugs filed by hackerone, instead of fire fighting these let's consider what it would take to refactor this nerdpack to make it more inline with modern security standards.

Usage

Acceptance Criteria

Update nr1-github

Acceptance Criteria

Upgrade to node 16
If applicable resolve any app deprecations planned for march 1st.
- You are not implementing property "NerdletSchema.actionCategory" as specified in the docs
- You are implementing a deprecated property "NerdletSchema.entities", please use "NerdletSchema.context.entities" instead
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

Update nr1-network-telemetry

Acceptance Criteria

Upgrade to node 16
If applicable resolve any app deprecations planned for march 1st.
- "normal" type for header controls has been deprecated. Use any of "primary", "secondary", "tertiary" instead
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

Update nr1-metrics-aggregator

Acceptance Criteria

Upgrade to node 16
If applicable resolve any app deprecations planned for march 1st.
- "errors" is deprecated, use "error?.graphQLErrors" instead
- "accountId" is deprecated, use "accountIds" instead
- "errors" is deprecated, use "error" instead
- "plain" type for header controls has been deprecated. Use any of "primary", "secondary", "tertiary" instead
update .nvrmc file to Node 16
Confirm if we need to set the NODE_VERSION environment variable
Ensure all documentation is updated.

newrelic / nr1-catalog Goto Github PK

nr1-catalog's Introduction

nr1-catalog

Usage

Use Case 1: PR with a new submodule

Use Case 2: PR to an existing submodule

Use Case 3: Merge of a PR to the master Catalog

Use Case 4: Removal of an Application from the NR1 Catalog

nr1-catalog's People

Contributors

Stargazers

Watchers

Forkers

nr1-catalog's Issues

Acceptance Criteria

Acceptance Criteria

Acceptance Criteria

Acceptance Criteria

Acceptance Criteria

Acceptance Criteria

Description

Apps

Acceptance Criteria

Description

AC

🏖️ 🌞 🥳 🍹

Acceptance Criteria

Description

Acceptance Criteria

Acceptance Criteria

Acceptance Criteria

Acceptance Criteria

Acceptance Criteria

Acceptance Criteria

Description

Apps

Summary

Acceptance Criteria

Problem

Acceptance Criteria

Summary

Acceptance Criteria

Affected repos

Acceptance Criteria

Usage

Acceptance Criteria

Epic Milestones

Epic Plan

Deprecation Dashboard for validation

Epic project board

Questions

Apps

Acceptance Criteria

Acceptance Criteria

Acceptance Criteria

Acceptance Criteria

Acceptance Criteria

Related issues

Usage

Acceptance Criteria

Acceptance Criteria

Acceptance Criteria

Acceptance Criteria

Recommend Projects

Recommend Topics

Recommend Org