newrelic / nri-mongodb Goto Github PK
View Code? Open in Web Editor NEWNew Relic Infrastructure MongoDB Integration
License: MIT License
New Relic Infrastructure MongoDB Integration
License: MIT License
Update documentation to point that mem.residentInBytes and mem.virtual are in MiB not Bytes.
When monitoring a sharded clusters in K8s the integration fails to discover the mongos
endpoint and no MongosSample
is generated.
Mongos metrics collection endpoints are discovered in the integration but in K8s the Id
on collections.config.mongos
contains the Pod name which cannot be resolved by the integration.
Support native mongos autodiscovery for K8s environments
Using bitnami mongo-sharded
chart and configured the integration with discovery to the mongos pod
nri-mongodb:
discovery:
command:
exec: /var/db/newrelic-infra/nri-discovery-kubernetes --port 10250 --tls
match:
label.app.kubernetes.io/component: mongos
integrations:
- name: nri-mongodb
env:
HOST: ${discovery.ip}
...
The integrations prints the error like:
[ERR] Failed to connect to mongos server <mongos-pod-name>:27017: no reachable servers
I workaround this by using creating a service in the Agent namespace that points to the mongos instance and which have the same name of the mongos pod (using StatefulSet kind to deploy mongos to have predictable pod names). But this was just for the propose of validate the issue.
When implementing the MongoDB integration for New Relic, I get an authentication failure when using any credentials using SCRAM. The same credentials works both in the MongoDB service and in the individual pods. I'm able to get all the databases and collections needed based on the role and user to add for New Relic to work properly.
The authentication should be working in the integration if it is working both on the service and the individual replicas.
time="2022-03-11T19:11:15Z" level=warning msg="integration exited with error state" component=integrations.runner.Runner env=azure error="exit status 1" integration_name=nri-mongodb stderr="[INFO] Collecting all collections for database 'test'\n[ERR] Failed to create session: server returned error on SASL authentication step: Authentication failed."
- name: mongodb-config.yml data: discovery: command: exec: /var/db/newrelic-infra/nri-discovery-kubernetes --namespaces mongodb --port 10250 --tls match: label.app: mongodb-replica-svc integrations: - name: nri-mongodb env: HOST: ${discovery.ip} PORT: 27017 USERNAME: newrelic PASSWORD: *** AUTH_SOURCE: admin MONGODB_CLUSTER_NAME: abc FILTERS: '{"test":null}' labels: env: dev
This fails, but mongodb://newrelic:***@localhost:27017/admin?authSource=admin&readPreference=primary&appname=MongoDB%20Compass&directConnection=true&ssl=false
works without any issues. The password is the same.
I'm using the latest helm chart version of the helm nri-bundle helm chart as well as the mongodb community operator. Kubernetes version is 1.21.
On multi-instance cluster the integration needs to point to one instance and metrics from all the instances are collected. This is a good feature but if the configured instance is down there will be no metrics collected.
Add a flag like MasterOnly
that if true
will only collect metrics if the current instance is the Master instance. With this flag, multiple instance of the integration can be configured to point all instances of the cluster.
Similar approach as nri-elasticsearch.
A clear and concise description of any alternative solutions or features you've considered. Are there examples you could link us to?
Add any other context here.
Please help us better understand this feature request by choosing a priority from the following options:
[Nice to Have, Really Want, Must Have, Blocker]
CPU and Memory Usage for nri-mongodb seems very high due to collectTop / MongoTopSample
Add flag to disable collectTop / MongoTopSample
We have added the nri-mongodb integration on our Servers. CPU and Memory Usage for nri-mongodb seems very high and the amount of data being sent to New Relic is also high.
Is there currently a way to disable collectTop that generates MongodTopSample? Or something someone can add easily? Or is this something we should look into adding and submitting a pull request for?
We are already excluding ingest of this data but dont want to keep collecting them on the server side as it seems to use alot of CPU and Mem resources.
We really only need Cluster Health related metrics, the rest we can debug manually on servers. Its nice to have on New Relic side but ingest amount seems extremely high and CPY and Mem usage also seem high.
Really Want
Our Documentation says: MongoDB Atlas is supported for tiers M10 and above.
However, it has been reported that in general we do not have enough documentation for that.
We should test the whole solution again, verify any limitation and improve the documentation.
๐คThis issue was automatically generated by repolinter-action, developed by the Open Source and Developer Advocacy team at New Relic. This issue will be automatically updated or closed when changes are pushed. If you have any problems with this tool, please feel free to open a GitHub issue or give us a ping in #help-opensource.
This Repolinter run generated the following results:
โ Error | โ Fail | โ Pass | Ignored | Total | |
---|---|---|---|---|---|
0 | 2 | 0 | 5 | 0 | 7 |
readme-starts-with-community-plus-header
#The README of a community plus project should have a community plus header at the start of the README. If you already have a community plus header and this rule is failing, your header may be out of date, and you should update your header with the suggested one below. For more information please visit https://opensource.newrelic.com/oss-category/. Below is a list of files or patterns that failed:
README.md
: The first 5 lines do not contain the pattern(s): Open source Community Plus header (see https://opensource.newrelic.com/oss-category).
[![Community Plus header](https://github.com/newrelic/opensource-website/raw/master/src/images/categories/Community_Plus.png)](https://opensource.newrelic.com/oss-category/#community-plus)
to filecode-of-conduct-should-not-exist-here
#New Relic has moved the CODE_OF_CONDUCT
file to a centralized location where it is referenced automatically by every repository in the New Relic organization. Because of this change, any other CODE_OF_CONDUCT
file in a repository is now redundant and should be removed. Note that you will need to adjust any links to the local CODE_OF_CONDUCT
file in your documentation to point to the central file (README
and CONTRIBUTING
will probably have links that need updating). For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view. Found files. Below is a list of files or patterns that failed:
CODE_OF_CONDUCT.md
license-file-exists
#Found file (LICENSE
). New Relic requires that all open source projects have an associated license contained within the project. This license must be permissive (e.g. non-viral or copyleft), and we recommend Apache 2.0 for most use cases. For more information please visit https://docs.google.com/document/d/1vML4aY_czsY0URu2yiP3xLAKYufNrKsc7o4kjuegpDw/edit.
readme-file-exists
#Found file (README.md
). New Relic requires a README file in all projects. This README should give a general overview of the project, and should point to additional resources (security, contributing, etc.) where developers and users can learn further. For more information please visit https://github.com/newrelic/open-by-default.
readme-contains-link-to-security-policy
#Contains a link to the security policy for this repository (README.md
). New Relic recommends putting a link to the open source security policy for your project (https://github.com/newrelic/<repo-name>/security/policy
or ../../security/policy
) in the README. For an example of this, please see the "a note about vulnerabilities" section of the Open By Default repository. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.
readme-contains-discuss-topic
#Contains a link to the appropriate discuss.newrelic.com topic (README.md
). New Relic recommends directly linking the your appropriate discuss.newrelic.com topic in the README, allowing developer an alternate method of getting support. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.
third-party-notices-file-exists
#Found file (THIRD_PARTY_NOTICES.md
). A THIRD_PARTY_NOTICES.md
file can be present in your repository to grant attribution to all dependencies being used by this project. This document is necessary if you are using third-party source code in your project, with the exception of code referenced outside the project's compiled/bundled binary (ex. some Java projects require modules to be pre-installed in the classpath, outside the project binary and therefore outside the scope of the THIRD_PARTY_NOTICES
). Please review your project's dependencies and create a THIRD_PARTY_NOTICES.md file if necessary. For JavaScript projects, you can generate this file using the oss-cli. For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view.
Currently we are using a third party mongodb library , we could move to the official one https://github.com/mongodb/mongo-go-driver
๐คThis issue was automatically generated by repolinter-action, developed by the Open Source and Developer Advocacy team at New Relic. This issue will be automatically updated or closed when changes are pushed. If you have any problems with this tool, please feel free to open a GitHub issue or give us a ping in #help-opensource.
This Repolinter run generated the following results:
โ Error | โ Fail | โ Pass | Ignored | Total | |
---|---|---|---|---|---|
0 | 0 | 0 | 7 | 0 | 7 |
license-file-exists
#Found file (LICENSE
). New Relic requires that all open source projects have an associated license contained within the project. This license must be permissive (e.g. non-viral or copyleft), and we recommend Apache 2.0 for most use cases. For more information please visit https://docs.google.com/document/d/1vML4aY_czsY0URu2yiP3xLAKYufNrKsc7o4kjuegpDw/edit.
readme-file-exists
#Found file (README.md
). New Relic requires a README file in all projects. This README should give a general overview of the project, and should point to additional resources (security, contributing, etc.) where developers and users can learn further. For more information please visit https://github.com/newrelic/open-by-default.
readme-starts-with-community-plus-header
#The first 5 lines contain all of the requested patterns. (README.md
). The README of a community plus project should have a community plus header at the start of the README. If you already have a community plus header and this rule is failing, your header may be out of date, and you should update your header with the suggested one below. For more information please visit https://opensource.newrelic.com/oss-category/.
readme-contains-link-to-security-policy
#Contains a link to the security policy for this repository (README.md
). New Relic recommends putting a link to the open source security policy for your project (https://github.com/newrelic/<repo-name>/security/policy
or ../../security/policy
) in the README. For an example of this, please see the "a note about vulnerabilities" section of the Open By Default repository. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.
readme-contains-forum-topic
#Contains a link to the appropriate forum.newrelic.com topic (README.md
). New Relic recommends directly linking the your appropriate forum.newrelic.com topic in the README, allowing developer an alternate method of getting support. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.
code-of-conduct-should-not-exist-here
#New Relic has moved the CODE_OF_CONDUCT
file to a centralized location where it is referenced automatically by every repository in the New Relic organization. Because of this change, any other CODE_OF_CONDUCT
file in a repository is now redundant and should be removed. Note that you will need to adjust any links to the local CODE_OF_CONDUCT
file in your documentation to point to the central file (README
and CONTRIBUTING
will probably have links that need updating). For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view. Did not find a file matching the specified patterns. All files passed this test.
third-party-notices-file-exists
#Found file (THIRD_PARTY_NOTICES.md
). A THIRD_PARTY_NOTICES.md
file can be present in your repository to grant attribution to all dependencies being used by this project. This document is necessary if you are using third-party source code in your project, with the exception of code referenced outside the project's compiled/bundled binary (ex. some Java projects require modules to be pre-installed in the classpath, outside the project binary and therefore outside the scope of the THIRD_PARTY_NOTICES
). Please review your project's dependencies and create a THIRD_PARTY_NOTICES.md file if necessary. For JavaScript projects, you can generate this file using the oss-cli. For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view.
The infra agent now supports:
mongodb+srv://
The globalsign/mbo
driver doesn't support mongodb+srv://
seedlists. Related - request for srv
support in the driver:
globalsign/mgo#112
Would there be an issue with switching from globalsign/mbo
to the driver published by MongoDB, which already has support for the new connection string format?
When make a new release, generate build for any linux distribution for easy install.
Create mapping on the metadata backend service infra backend
When the keys from the top
command are not of the form <database>.<collection>
, the integration fails because it attempts to split on .
and take indices 0 and 1. We should handle this more gracefully, either logging and skipping or fililng in the database and collection with dummy values.
src/entities/collect.go
174 for key, collectionStats := range topMetrics.Totals {
175 splitKey := strings.SplitN(key, ".", 2)
176 databaseName := splitKey[0]
177 collectionName := splitKey[1]
178
179 ms := e.NewMetricSet("MongodTopSample",
180 metric.Attribute{Key: "displayName", Value: e.Metadata.Name},
181 metric.Attribute{Key: "entityName", Value: fmt.Sprintf("%s:%s", e.Metadata.Namespace, e.Metadata.Name)},
182 metric.Attribute{Key: "database", Value: databaseName},
183 metric.Attribute{Key: "collection", Value: collectionName},
184 )
185
186 logError(ms.MarshalMetrics(collectionStats), "Marshal metrics on top failed: %v")
187
188 }
Working with a customer and they had to re-configure their Atlas implementation to allow a public endpoint connection. The New Relic MongoDB integration evidently will not work with the DNS seed config where SRV: becomes part of the connection string for a private endpoint.
Improve static analysis with Semgrep adding the newrelic-infra-checkers actions to get the semgrep and golangci-lint configurations.
๐คThis issue was automatically generated by repolinter-action, developed by the Open Source and Developer Advocacy team at New Relic. This issue will be automatically updated or closed when changes are pushed. If you have any problems with this tool, please feel free to open a GitHub issue or give us a ping in #help-opensource.
This Repolinter run generated the following results:
โ Error | โ Fail | โ Pass | Ignored | Total | |
---|---|---|---|---|---|
0 | 0 | 0 | 7 | 0 | 7 |
license-file-exists
#Found file (LICENSE
). New Relic requires that all open source projects have an associated license contained within the project. This license must be permissive (e.g. non-viral or copyleft), and we recommend Apache 2.0 for most use cases. For more information please visit https://docs.google.com/document/d/1vML4aY_czsY0URu2yiP3xLAKYufNrKsc7o4kjuegpDw/edit.
readme-file-exists
#Found file (README.md
). New Relic requires a README file in all projects. This README should give a general overview of the project, and should point to additional resources (security, contributing, etc.) where developers and users can learn further. For more information please visit https://github.com/newrelic/open-by-default.
readme-starts-with-community-plus-header
#The first 5 lines contain all of the requested patterns. (README.md
). The README of a community plus project should have a community plus header at the start of the README. If you already have a community plus header and this rule is failing, your header may be out of date, and you should update your header with the suggested one below. For more information please visit https://opensource.newrelic.com/oss-category/.
readme-contains-link-to-security-policy
#Contains a link to the security policy for this repository (README.md
). New Relic recommends putting a link to the open source security policy for your project (https://github.com/newrelic/<repo-name>/security/policy
or ../../security/policy
) in the README. For an example of this, please see the "a note about vulnerabilities" section of the Open By Default repository. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.
readme-contains-discuss-topic
#Contains a link to the appropriate discuss.newrelic.com topic (README.md
). New Relic recommends directly linking the your appropriate discuss.newrelic.com topic in the README, allowing developer an alternate method of getting support. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.
code-of-conduct-should-not-exist-here
#New Relic has moved the CODE_OF_CONDUCT
file to a centralized location where it is referenced automatically by every repository in the New Relic organization. Because of this change, any other CODE_OF_CONDUCT
file in a repository is now redundant and should be removed. Note that you will need to adjust any links to the local CODE_OF_CONDUCT
file in your documentation to point to the central file (README
and CONTRIBUTING
will probably have links that need updating). For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view. Did not find a file matching the specified patterns. All files passed this test.
third-party-notices-file-exists
#Found file (THIRD_PARTY_NOTICES.md
). A THIRD_PARTY_NOTICES.md
file can be present in your repository to grant attribution to all dependencies being used by this project. This document is necessary if you are using third-party source code in your project, with the exception of code referenced outside the project's compiled/bundled binary (ex. some Java projects require modules to be pre-installed in the classpath, outside the project binary and therefore outside the scope of the THIRD_PARTY_NOTICES
). Please review your project's dependencies and create a THIRD_PARTY_NOTICES.md file if necessary. For JavaScript projects, you can generate this file using the oss-cli. For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view.
I get following error in logs
Failed to create session: server returned error on SASL authentication step: BSON field 'saslContinue.mechanism' is an unknown field."
I think the problem is with old drivers.
Currently, the integration litters the logs with errors when a standalone instance is monitored rather than a cluster. Additionally, documentation doesn't clearly describe the differences in metric collection when monitoring a standalone instance.
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These updates have all been created already. Click a checkbox below to force a retry/rebase of any.
tests/integration/docker-compose.yml
docker.io/bitnami/mongodb-sharded 3.6
docker.io/bitnami/mongodb-sharded 3.6
docker.io/bitnami/mongodb-sharded 3.6
build/Dockerfile
golang 1.22.5-bookworm
tests/integration/Dockerfile
golang 1.22.5-bookworm
.github/workflows/automated_release.yaml
newrelic/coreint-automation v3
.github/workflows/on_prerelease.yml
newrelic/coreint-automation v3
.github/workflows/on_push_pr.yml
newrelic/coreint-automation v3
.github/workflows/on_release.yml
newrelic/coreint-automation v3
.github/workflows/repolinter.yml
newrelic/coreint-automation v3
.github/workflows/security.yml
newrelic/coreint-automation v3
go.mod
go 1.22.5
github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8@eeefdecb41b8
github.com/newrelic/infra-integrations-sdk v3.8.2+incompatible
github.com/stretchr/testify v1.9.0
github.com/xeipuuv/gojsonschema v1.2.0
๐คThis issue was automatically generated by repolinter-action, developed by the Open Source and Developer Advocacy team at New Relic. This issue will be automatically updated or closed when changes are pushed. If you have any problems with this tool, please feel free to open a GitHub issue or give us a ping in #help-opensource.
This Repolinter run generated the following results:
โ Error | โ Fail | โ Pass | Ignored | Total | |
---|---|---|---|---|---|
0 | 1 | 0 | 6 | 0 | 7 |
readme-starts-with-community-plus-header
#The README of a community plus project should have a community plus header at the start of the README. If you already have a community plus header and this rule is failing, your header may be out of date, and you should update your header with the suggested one below. For more information please visit https://opensource.newrelic.com/oss-category/. Below is a list of files or patterns that failed:
README.md
: The first 5 lines do not contain the pattern(s): Open source Community Plus header (see https://opensource.newrelic.com/oss-category).
[![Community Plus header](https://github.com/newrelic/opensource-website/raw/master/src/images/categories/Community_Plus.png)](https://opensource.newrelic.com/oss-category/#community-plus)
to filelicense-file-exists
#Found file (LICENSE
). New Relic requires that all open source projects have an associated license contained within the project. This license must be permissive (e.g. non-viral or copyleft), and we recommend Apache 2.0 for most use cases. For more information please visit https://docs.google.com/document/d/1vML4aY_czsY0URu2yiP3xLAKYufNrKsc7o4kjuegpDw/edit.
readme-file-exists
#Found file (README.md
). New Relic requires a README file in all projects. This README should give a general overview of the project, and should point to additional resources (security, contributing, etc.) where developers and users can learn further. For more information please visit https://github.com/newrelic/open-by-default.
readme-contains-link-to-security-policy
#Contains a link to the security policy for this repository (README.md
). New Relic recommends putting a link to the open source security policy for your project (https://github.com/newrelic/<repo-name>/security/policy
or ../../security/policy
) in the README. For an example of this, please see the "a note about vulnerabilities" section of the Open By Default repository. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.
readme-contains-discuss-topic
#Contains a link to the appropriate discuss.newrelic.com topic (README.md
). New Relic recommends directly linking the your appropriate discuss.newrelic.com topic in the README, allowing developer an alternate method of getting support. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.
code-of-conduct-should-not-exist-here
#New Relic has moved the CODE_OF_CONDUCT
file to a centralized location where it is referenced automatically by every repository in the New Relic organization. Because of this change, any other CODE_OF_CONDUCT
file in a repository is now redundant and should be removed. Note that you will need to adjust any links to the local CODE_OF_CONDUCT
file in your documentation to point to the central file (README
and CONTRIBUTING
will probably have links that need updating). For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view. Did not find a file matching the specified patterns. All files passed this test.
third-party-notices-file-exists
#Found file (THIRD_PARTY_NOTICES.md
). A THIRD_PARTY_NOTICES.md
file can be present in your repository to grant attribution to all dependencies being used by this project. This document is necessary if you are using third-party source code in your project, with the exception of code referenced outside the project's compiled/bundled binary (ex. some Java projects require modules to be pre-installed in the classpath, outside the project binary and therefore outside the scope of the THIRD_PARTY_NOTICES
). Please review your project's dependencies and create a THIRD_PARTY_NOTICES.md file if necessary. For JavaScript projects, you can generate this file using the oss-cli. For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view.
The MongodTopSample
is missing from documentation.
The default dashboard for MongoDB in NROne has several widgets querying against MongoSample
, which results in empty/blank charts.
Navigate to Infrastructure
>> Third-party services
>> MongoDB Dashboard
in NROne
I'm try to integrate MR and MongoDB atlas by using this repo. I have renamed mongodb-atlas-config.yml and added required configurations. But getting below error when try to execute.
[ERR] Invalid arguments: Must supply a cluster name to identify this Mongodb cluster. Use MongodbClusterName config parameter
Could you please share guidance to configure
Thanks
Tharindu
Integration fails due to no mongo 4+ SCRAM-SHA-256 authentication support. I've found that the driver you all use has yet to enable support for SCRAM-SHA-256 authentication as well. There's an existing pr (stale - 2+ years old) in the driver repo you all use. I understand that this will need to be done first in order to enable here, so I'm curious and asking if the mgo feature is completed would you all be willing to support it.
Enable mongo 4+ SCRAM-SHA-256 authentication mechanism
n/a
Existing mgo pr seeking support for completion
Blocker
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.