The following document describes how to configure Datadog to gather logs, metrics and violations from Artifactory and Xray through the use of FluentD.
This integration is last tested with Artifactory 7.84.17 and Xray 3.92.7 versions.
Note! You must follow the order of the steps throughout Datadog Configuration
DataDog setup for this integration can be done by going through the step below to add a new DataDog apiKey or by using an existing DataDog apiKey.
If a DataDog apiKey already exists and can be used for this integration, skip this part and move on to Fluentd Installation to forward logs and metrics to your DataDog account.
If you don't have a DataDog apiKey:
- Create an account in DataDog, if one doesn't exist
- Follow the official DataDog instructions here how to generate an apiKey that will be used in the following sections
Metrics collection is disabled by default in Artifactory by default. For non-kubernetes installations, to enable metrics in Artifactory, make the following configuration changes to the Artifactory System YAML:
shared:
metrics:
enabled: true
artifactory:
metrics:
enabled: true
Once this configuration is done and the application is restarted, metrics will be available in Open Metrics Format
๐ก Metrics are enabled by default in Xray.
๐ก For kubernetes based installs, openMetrics collection is enabled in the helm install commands listed in the sections below
Ensure you have access to the Internet from a virtual machine (VM). We recommend installation through FluentD's native OS based package installs:
OS | Package Manager | Link |
---|---|---|
CentOS/RHEL | Linux - RPM (YUM) | https://docs.fluentd.org/installation/install-by-rpm |
Debian/Ubuntu | Linux - APT | https://docs.fluentd.org/installation/install-by-deb |
MacOS/Darwin | MacOS - DMG | https://docs.fluentd.org/installation/install-by-dmg |
Windows | Windows - MSI | https://docs.fluentd.org/installation/install-by-msi |
Gem Install** | MacOS & Linux - Gem | https://docs.fluentd.org/installation/install-by-gem |
For a Gem-based install, the Ruby Interpreter must be setup first. You can install the Ruby Interpreter by doing the following:
-
Install Ruby Version Manager (RVM) outlined in the RVM documentation.
- Use the
SUDO
command for multi-user installation. For more information, see the RVM troubleshooting documentation.
- Use the
-
After the RVM installation is complete, execute the command 'rvm -v' to verify.
-
Install Ruby v3.3.0 or above with the command
rvm install <ver_num>
, (for example,rvm install 3.3.0
). -
Verify the Ruby installation, execute
ruby -v
, gem installationgem -v
andbundler -v
to ensure all the components are intact. -
Install the FluentD gem with the command
gem install fluentd
. -
After FluentD is successfully installed, install the following plugins.
gem install fluent-plugin-concat
gem install fluent-plugin-datadog
gem install fluent-plugin-jfrog-siem
gem install fluent-plugin-jfrog-metrics
gem install fluent-plugin-jfrog-send-metrics
We rely on environment variables to stream log files to your observability dashboards. Ensure that you fill in the .env
file with the correct values. You can download the .env
file here.
- JF_PRODUCT_DATA_INTERNAL: The environment variable JF_PRODUCT_DATA_INTERNAL must be defined to the correct location. For each JFrog service, you can find its active log files in the
$JFROG_HOME/<product>/var/log
directory - DATADOG_API_KEY: API Key from Datadog
- DATADOG_API_HOST: Your DataDog host based on your DataDog Site Parameter from this list
- JPD_URL: Artifactory JPD URL with the format
http://<ip_address>
- JPD_ADMIN_USERNAME: Artifactory username for authentication
- JFROG_ADMIN_TOKEN: Artifactory Access Token for authentication
- COMMON_JPD: This flag should be set as true only for non-Kubernetes installations or installations where the JPD base URL is the same to access both Artifactory and Xray (for example,
https://sample_base_url/artifactory
orhttps://sample_base_url/xray
)
Apply the .env
files and run the fluentd wrapper with the following command, and note that the argument points to the fluent.conf.*
file previously configured:
source jfrog.env
./fluentd $JF_PRODUCT_DATA_INTERNAL/fluent.conf.<product_name>
In order to run FluentD as a docker image to send the logs, violations, and metrics data to Datadog, execute the following commands on the host that runs the docker.
-
Execute the
docker version
anddocker ps
commands to verify that the Docker installation is functional. -
If the version and process are listed successfully, build the intended docker image for Datadog using the docker file. You can download [this Dockerfile]https://raw.githubusercontent.com/jfrog/log-analytics-datadog/master/docker-build/Dockerfile to any directory that has write permissions.
-
Download the
docker.env
file needed to runJfrog/FluentD
Docker Images for Datadog. You can download [this docker.env]https://raw.githubusercontent.com/jfrog/log-analytics-datadog/master/docker-build/docker.env to the directory where the docker file was downloaded. -
Execute the following command to build the docker image:
docker build --build-arg SOURCE="JFRT" --build-arg TARGET="DATADOG" -t <image_name>
. For example:docker build --build-arg SOURCE="JFRT" --build-arg TARGET="DATADOG" -t jfrog/fluentd-datadog-rt .'
-
Fill out the necessary information in the docker.env file:
- JF_PRODUCT_DATA_INTERNAL: The environment variable JF_PRODUCT_DATA_INTERNAL must be defined to the correct location. For each JFrog service you will find its active log files in the
$JFROG_HOME/<product>/var/log
directory - DATADOG_API_KEY: API Key from Datadog
- DATADOG_API_HOST: Your DataDog host based on your DataDog Site Parameter from this list
- JPD_URL: Artifactory JPD URL of the format
http://<ip_address>
- JPD_ADMIN_USERNAME: Artifactory username for authentication
- JFROG_ADMIN_TOKEN: Artifactory Access Token for authentication
- COMMON_JPD: This flag should be set as true only for non-kubernetes installations or installations where JPD base URL is same to access both Artifactory and Xray (ex: https://sample_base_url/artifactory or https://sample_base_url/xray)
- JF_PRODUCT_DATA_INTERNAL: The environment variable JF_PRODUCT_DATA_INTERNAL must be defined to the correct location. For each JFrog service you will find its active log files in the
-
Execute 'docker run -it --name jfrog-fluentd-datadog-rt -v <path_to_logs>:/var/opt/jfrog/artifactory --env-file docker.env <image_name>'
The
<path_to_logs>
should be an absolute path where the Jfrog Artifactory Logs folder resides, such as a Docker based Artifactory Installation like/var/opt/jfrog/artifactory/var/logs
on the docker host. For example:docker run -it --name jfrog-fluentd-datadog-rt -v $JFROG_HOME/artifactory/var/:/var/opt/jfrog/artifactory --env-file docker.env jfrog/fluentd-datadog-rt
The recommended installation method for Kubernetes is to utilize the helm chart with the associated values.yaml in this repo.
Product | Example Values File |
---|---|
Artifactory | helm/artifactory-values.yaml |
Artifactory HA | helm/artifactory-ha-values.yaml |
Xray | helm/xray-values.yaml |
Warning
The old docker registry partnership-pts-observability.jfrog.io
, which contains older versions of this integration is now deprecated. We'll keep the existing docker images on this old registry until August 1st, 2024. After that date, this registry will no longer be available. Please helm upgrade
your JFrog kubernetes deployment in order to pull images as specified on the above helm value files, from the new releases-pts-observability-fluentd.jfrog.io
registry. Please do so in order to avoid ImagePullBackOff
errors in your deployment once this registry is gone.
Add JFrog Helm repository:
helm repo add jfrog https://charts.jfrog.io
helm repo update
Throughout the exampled helm installations we'll use jfrog-dd
as an example namespace. That said, you can use a different or existing namespace instead by setting the following environment variable
export INST_NAMESPACE=jfrog-dd
If you don't have an existing namespace for the deployment, create it and set the kubectl context to use this namespace
kubectl create namespace $INST_NAMESPACE
kubectl config set-context --current --namespace=$INST_NAMESPACE
Generate masterKey
and joinKey
for the installation
export JOIN_KEY=$(openssl rand -hex 32)
export MASTER_KEY=$(openssl rand -hex 32)
-
Skip this step if you already have Artifactory installed. Else, install Artifactory using the command below
helm upgrade --install artifactory jfrog/artifactory \ --set artifactory.masterKey=$MASTER_KEY \ --set artifactory.joinKey=$JOIN_KEY \ --set artifactory.license.secret=artifactory-license \ --set artifactory.license.dataKey=artifactory.cluster.license \ --set artifactory.metrics.enabled=true \ -n $INST_NAMESPACE --create-namespace
๐ก Metrics collection is disabled by default in Artifactory. Please make sure that you are following the above
helm upgrade
command to enable them in Artifactory by settingartifactory.metrics.enabled=true
. For Artifactory versions <=7.86.x, please enable metrics by setting the flagartifactory.openMetrics.enabled=true
Get the ip address of the newly deployed Artifactory:
export SERVICE_IP=$(kubectl get svc -n $INST_NAMESPACE artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}') echo $SERVICE_IP
-
Create a secret for JFrog's admin token - Access Token using any of the following methods
kubectl create secret generic jfrog-admin-token --from-file=token=<path_to_token_file> OR kubectl create secret generic jfrog-admin-token --from-literal=token=<JFROG_ADMN_TOKEN>
-
For Artifactory installation, download the .env file from here. Fill in the jfrog_helm.env file with correct values.
- JF_PRODUCT_DATA_INTERNAL: Helm based installs will already have this defined based upon the underlying Docker images. Not a required field for k8s installation
- DATADOG_API_KEY: API Key from Datadog
- DATADOG_API_HOST: Your DataDog host based on your DataDog Site Parameter from this list
- JPD_URL: Artifactory JPD URL of the format
http://<ip_address>
- JPD_ADMIN_USERNAME: Artifactory username for authentication
- COMMON_JPD: This flag should be set as true only for non-Kubernetes installations or installations where the JPD base URL is the same to access both Artifactory and Xray (for example,
https://sample_base_url/artifactory
orhttps://sample_base_url/xray
)
Apply the .env files using the helm command below
source jfrog_helm.env
-
Postgres password is required to upgrade Artifactory. Run the following command to get the current password
POSTGRES_PASSWORD=$(kubectl get secret artifactory-postgresql -n $INST_NAMESPACE -o jsonpath="{.data.postgresql-password}" | base64 --decode)
-
Upgrade Artifactory installation using the command below
helm upgrade --install artifactory jfrog/artifactory \ --set artifactory.joinKey=$JOIN_KEY \ --set databaseUpgradeReady=true --set postgresql.postgresqlPassword=$POSTGRES_PASSWORD --set nginx.service.ssloffload=true \ --set datadog.api_key=$DATADOG_API_KEY \ --set datadog.api_host=$DATADOG_API_HOST \ --set jfrog.observability.jpd_url=$JPD_URL \ --set jfrog.observability.username=$JPD_ADMIN_USERNAME \ --set jfrog.observability.common_jpd=$COMMON_JPD \ -f helm/artifactory-values.yaml \ -n $INST_NAMESPACE
-
For HA installation, please create a license secret on your cluster prior to installation.
kubectl create secret generic artifactory-license --from-file=<path_to_license_file>artifactory.cluster.license
-
Skip this step if you already have Artifactory installed. Else, install Artifactory using the command below
helm upgrade --install artifactory-ha jfrog/artifactory-ha \ --set artifactory.masterKey=$MASTER_KEY \ --set artifactory.joinKey=$JOIN_KEY \ --set artifactory.license.secret=artifactory-license \ --set artifactory.license.dataKey=artifactory.cluster.license \ --set artifactory.metrics.enabled=true \ -n $INST_NAMESPACE
๐ก Metrics collection is disabled by default in Artifactory-HA. Please make sure that you are following the above
helm upgrade
command to enable them in Artifactory by settingartifactory.metrics.enabled=true
. For Artifactory versions <=7.86.x, please enable metrics by setting the flagartifactory.openMetrics.enabled=true
Get the ip address of the newly deployed Artifactory:
export SERVICE_IP=$(kubectl get svc -n $INST_NAMESPACE artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}') echo $SERVICE_IP
-
Create a secret for JFrog's admin token - Access Token using any of the following methods
kubectl create secret generic jfrog-admin-token --from-file=token=<path_to_token_file> OR kubectl create secret generic jfrog-admin-token --from-literal=token=<JFROG_ADMN_TOKEN>
-
Download the .env file from here. Fill in the jfrog_helm.env file with correct values.
- JF_PRODUCT_DATA_INTERNAL: Helm based installs will already have this defined based upon the underlying Docker images. Not a required field for k8s installation
- DATADOG_API_KEY: API Key from Datadog
- DATADOG_API_HOST: Your DataDog host based on your DataDog Site Parameter from this list
- JPD_URL: Artifactory JPD URL of the format
http://<ip_address>
- JPD_ADMIN_USERNAME: Artifactory username for authentication
- COMMON_JPD: This flag should be set as true only for non-Kubernetes installations or installations where the JPD base URL is the same to access both Artifactory and Xray (for example,
https://sample_base_url/artifactory
orhttps://sample_base_url/xray
)
Apply the .env files and then run the helm command below
source jfrog_helm.env
-
Postgres password is required to upgrade Artifactory. Run the following command to get the current password
POSTGRES_PASSWORD=$(kubectl get secret artifactory-ha-postgresql -n $INST_NAMESPACE -o jsonpath="{.data.postgresql-password}" | base64 --decode)
-
Upgrade Artifactory HA installation using the command below
helm upgrade --install artifactory-ha jfrog/artifactory-ha \ --set artifactory.joinKey=$JOIN_KEY \ --set artifactory.openMetrics.enabled=true \ --set databaseUpgradeReady=true --set postgresql.postgresqlPassword=$POSTGRES_PASSWORD --set nginx.service.ssloffload=true \ --set datadog.api_key=$DATADOG_API_KEY \ --set datadog.api_host=$DATADOG_API_HOST \ --set jfrog.observability.jpd_url=$JPD_URL \ --set jfrog.observability.username=$JPD_ADMIN_USERNAME \ --set jfrog.observability.common_jpd=$COMMON_JPD \ -f helm/artifactory-ha-values.yaml \ -n $INST_NAMESPACE
Create a secret for JFrog's admin token - Access Token using any of the following methods if it doesn't exist
kubectl create secret generic jfrog-admin-token --from-file=token=<path_to_token_file>
OR
kubectl create secret generic jfrog-admin-token --from-literal=token=<JFROG_ADMN_TOKEN>
For Xray installation, download the .env file from here. Fill in the jfrog_helm.env file with correct values.
- JF_PRODUCT_DATA_INTERNAL: Helm based installs will already have this defined based upon the underlying Docker images. Not a required field for k8s installation
- DATADOG_API_KEY: API Key from Datadog
- JPD_URL: Artifactory JPD URL of the format
http://<ip_address>
- JPD_ADMIN_USERNAME: Artifactory username for authentication
- COMMON_JPD: This flag should be set as true only for non-Kubernetes installations or installations where the JPD base URL is the same to access both Artifactory and Xray (for example,
https://sample_base_url/artifactory
orhttps://sample_base_url/xray
)
Apply the .env files and then run the helm command below
source jfrog_helm.env
Generate a master key for xray
export XRAY_MASTER_KEY=$(openssl rand -hex 32)
Use the same joinKey
as you used in Artifactory installation to allow Xray node to successfully connect to Artifactory.
helm upgrade --install xray jfrog/xray --set xray.jfrogUrl=$JPD_URL \
--set xray.masterKey=$XRAY_MASTER_KEY \
--set xray.joinKey=$JOIN_KEY \
--set datadog.api_key=$DATADOG_API_KEY \
--set datadog.api_host=$DATADOG_API_HOST \
--set jfrog.observability.jpd_url=$JPD_URL \
--set jfrog.observability.username=$JPD_ADMIN_USERNAME \
--set jfrog.observability.common_jpd=$COMMON_JPD \
-f helm/xray-values.yaml \
-n $INST_NAMESPACE
This dashboard is divided into three sections Application, Audit and Requests
- Application - This section tracks Log Volume(information about different log sources) and Artifactory Errors over time(bursts of application errors that may otherwise go undetected)
- Audit - This section tracks audit logs help you determine who is accessing your Artifactory instance and from where. These can help you track potentially malicious requests or processes (such as CI jobs) using expired credentials.
- Requests - This section tracks HTTP response codes, Top 10 IP addresses for uploads and downloads
This dashboard tracks Artifactory System Metrics, JVM memory, Garbabe Collection, Database Connections, and HTTP Connections metrics
This dashboard provides a summary of access, service and traffic log volumes associated with Xray. Additionally, customers are also able to track various HTTP response codes, HTTP 500 errors, and log errors for greater operational insight
This dashboard provides an aggregated summary of all the license violations and security vulnerabilities found by Xray. Information is segment by watch policies and rules. Trending information is provided on the type and severity of violations over time, as well as, insights on most frequently occurring CVEs, top impacted artifacts and components.
This dashboard tracks System Metrics, and data metrics about Scanned Artifacts and Scanned Components
- Kubernetes Cluster
- Artifactory and/or Xray installed via JFrog Helm Charts
- Helm 3
Partner Integration Test Framework can be used to generate data for metrics.
- Datadog - Cloud monitoring as a service