dynatrace / dynatrace-appmon-puppet Goto Github PK

View Code? Open in Web Editor NEW

20.0 45.0 12.0 279 KB

Installs the Dynatrace Application Monitoring solution using Puppet.

License: MIT License

Ruby 32.91% Puppet 44.45% Shell 18.62% HTML 4.03%

appmon deployment

dynatrace-appmon-puppet's Introduction

dynatrace Module

This Puppet module installs and configures the Dynatrace Application Monitoring solution.

Requirements

Please see Puppetfile and metadata.json for a list of module dependencies.

Classes

dynatraceappmon::role::agents_package

Installs the Dynatrace Agents package.

This class downloads and installs the most recent version of the Dynatrace Agents package for the Linux platform from http://downloads.dynatracesaas.com. The default behavior can be overridden via the class' $installer_file_url parameter. Alternatively, you can place the installer artifact as dynatrace-agent.jar in the module's files directory from where it will be picked up during the installation. Please refer to manifests/role/agents_package.pp for a list of supported parameters, whose default values can be overridden in manifests/params.pp. In order to install, you may execute the class as follows:

class { 'dynatraceappmon::role::agents_package':
  installer_file_url => 'https://files.dynatrace.com/downloads/OnPrem/dynaTrace/7.2/7.2.0.1697/dynatrace-agent-7.2.0.1697-unix.jar'
}

Note: this class merely makes the Dynatrace Agents available, but it does not configure your application to actually load any. See the dynatraceappmon::role::java_agent class for an example that does.

dynatraceappmon::role::apache_wsagent

Installs the Dynatrace WebServer Agent for the Apache HTTP Server.

Please refer to manifests/role/apache_wsagent.pp and manifests/role/wsagent_package.pp for a list of supported parameters, whose default values can be overridden in manifests/params.pp. In order to install, you may execute the class as follows:

class { 'dynatraceappmon::role::wsagent_package':
  installer_file_url => 'https://files.dynatrace.com/downloads/OnPrem/dynaTrace/7.2/7.2.0.1697/dynatrace-wsagent-7.2.0.1697-linux-x86-64.tar'
}

class { 'dynatraceappmon::role::apache_wsagent':
  apache_config_file_path => '/etc/apache2/apache2.conf',
  require                 => Class['dynatraceappmon::role::wsagent_package']
}

Note: you will have to restart the web server after placing the agent.

dynatraceappmon::role::collector

Installs the Dynatrace Collector.

This class downloads and installs the most recent version of the Dynatrace Collector for the Linux platform from http://downloads.dynatracesaas.com. The default behavior can be overridden via the class' $installer_file_url parameter. Alternatively, you can place the installer artifact as dynatrace-collector.jar in the module's files directory from where it will be picked up during the installation. Please refer to manifests/role/collector.pp for a list of supported parameters, whose default values can be overridden in manifests/params.pp. In order to install, you may execute the class as follows:

class { 'dynatraceappmon::role::collector':
  installer_file_url => 'https://files.dynatrace.com/downloads/OnPrem/dynaTrace/7.2/7.2.0.1697/dynatrace-collector-7.2.0.1697-linux-x86.jar'
}

Note: make sure that attributes related to the Collector's memory configuration are set in accordance to the Memory Configuration documentation.

dynatraceappmon::role::java_agent

Installs the Dynatrace Java Agent.

Please refer to manifests/role/java_agent.pp and manifests/role/agents_package.pp for a list of supported parameters, whose default values can be overridden in manifests/params.pp. In order to install, you may execute the class as follows:

class { 'dynatraceappmon::role::agents_package':
  installer_file_url => 'https://files.dynatrace.com/downloads/OnPrem/dynaTrace/7.2/7.2.0.1697/dynatrace-agent-7.2.0.1697-unix.jar'
}

class { 'dynatraceappmon::role::java_agent':
  env_var_name      => 'CATALINA_OPTS',
  env_var_file_name => '/opt/apache-tomcat/bin/catalina.sh',
  agent_name        => 'apache-tomcat-agent',
  require           => Class['dynatraceappmon::role::agents_package']
}

Note: this recipe makes the Java Agent available to a Java Virtual Machine by injecting an appropriate -agentpath option into an environment variable, e.g. JAVA_OPTS, inside a file (typically an executable script). It is assumed that this script either executes the Java process directly or is sourced by another script before the Java process gets executed. You will have to restart the application after placing the agent.

dynatraceappmon::role::memory_analysis_server

Installs the Dynatrace Memory Analysis Server.

This class downloads and installs the most recent version of the Dynatrace Memory Analysis Server for the Linux platform from http://downloads.dynatracesaas.com. The default behavior can be overridden via the class' $installer_file_url parameter. Alternatively, you can place the installer artifact as dynatrace-analysisserver.jar in the module's files directory from where it will be picked up during the installation. Please refer to manifests/role/memory_analysis_server.pp for a list of supported parameters, whose default values can be overridden in manifests/params.pp. In order to install, you may execute the class as follows:

class { 'dynatraceappmon::role::memory_analysis_server':
  installer_file_url => 'https://files.dynatrace.com/downloads/OnPrem/dynaTrace/7.2/7.2.0.1697/dynatrace-analysisserver-7.2.0.1697-linux-x86.jar'
}

Note: make sure that attributes related to the Analysis Server's memory configuration are set in accordance to the documentation Set up a Memory Analysis Server.

dynatraceappmon::role::php_one_agent

Installs the PHP OneAgent.

This class downloads and installs the most recent version of the PHP OneAgent for the Linux platform from http://downloads.dynatracesaas.com. The default behavior can be overridden via the class' $installer_file_url parameter. Alternatively, you can place the installer artifact as dynatrace-one-agent-php.tar in the module's files directory from where it will be picked up during the installation. Please refer to manifests/role/php_one_agent.pp for a list of supported parameters, whose default values can be overridden in manifests/params.pp. In order to install, you may execute the class as follows:

class { 'dynatraceappmon::role::php_one_agent':
  installer_file_url => 'https://files.dynatrace.com/downloads/OnPrem/dynaTrace/7.2/7.2.0.1697/dynatrace-one-agent-php-7.2.0.1697-linux-x86.tgz'
}

Note: make sure that attributes related to the One Agent configuration are set in accordance to the documentation PHP Agent configuration and OneAgent configuration.

dynatraceappmon::role::server

Installs the Dynatrace Server.

This class downloads and installs the most recent version of the Dynatrace Memory Analysis Server for the Linux platform from http://downloads.dynatracesaas.com. The default behavior can be overridden via the class' $installer_file_url parameter. Alternatively, you can place the installer artifact as dynatrace.jar in the module's files directory from where it will be picked up during the installation. Please refer to manifests/role/server.pp for a list of supported parameters, whose default values can be overridden in manifests/params.pp. In order to install, you may execute the class as follows:

class { 'dynatraceappmon::role::server':
  installer_file_url => 'https://files.dynatrace.com/downloads/OnPrem/dynaTrace/7.2/7.2.0.1697/dynatrace-full-7.2.0.1697-linux-x86-64.jar'
}

dynatraceappmon::role::server_license

Installs the Dynatrace Server License.

Place the Dynatrace Server License as dynatrace-license.key in the module's files directory. Alternatively, you can make the license available as an HTTP, HTTPS or FTP resource and point the class to the right location via the $license_file_url parameter. Please refer to manifests/role/server_license.pp for a list of supported attributes, whose default values can be overridden in manifests/params.pp. In order to install, you may execute the class as follows:

class { 'dynatraceappmon::role::server':
  license_file_url => 'http://my-license-server/dynatrace/dynatrace-license.key'
}

dynatraceappmon::role::wsagent_package

Installs the Dynatrace WebServer Agent package.

This class downloads and installs the most recent version of the Dynatrace WebServer Agent installer for the Linux platform from http://downloads.dynatracesaas.com. The default behavior can be overridden via the class' $installer_file_url parameter. Alternatively, you can place the installer artifact as dynatrace-wsagent.tar in the module's files directory from where it will be picked up during the installation. Please refer to manifests/role/wsagent_package.pp for a list of supported parameters, whose default values can be overridden in manifests/params.pp. In order to install, you may execute the class as follows:

class { 'dynatraceappmon::role::wsagent_package':
  installer_file_url => 'https://files.dynatrace.com/downloads/OnPrem/dynaTrace/7.2/7.2.0.1697/dynatrace-wsagent-7.2.0.1697-linux-x86-64.tar'
}

Note: this recipe merely makes the Dynatrace WebServer Agent available, but it does not configure your web server to actually load it. See the dynatraceappmon::role::apache_wsagent class for an example that does.

Testing

We use Test Kitchen to automatically test our automated deployments with Serverspec and RSpec:

Install Test Kitchen and its dependencies from within the project's directory:

gem install bundler
bundle install

Run all tests

kitchen test

By default, we run our tests inside Docker containers as this considerably speeds up testing time (see .kitchen.yml, requires Ruby 2.2+). Alternatively, you may as well run these tests in virtual machines based on VirtualBox and Vagrant (see .kitchen.vagrant.yml).

Additional Resources

Slide Deck: Test-Driven Infrastructure with Test Kitchen, Serverspec and RSpec

Questions?

Feel free to post your questions on the Dynatrace Community's Continuous Delivery Forum.

License

Licensed under the MIT License. See the LICENSE file for details.

dynatrace-appmon-puppet's People

Contributors

Stargazers

Watchers

Forkers

nespresso slantedvision tedfeyler brandonmb legal90 bardyr dansksupermarked jkulesz kdemusz will-wk-chan binford2k

dynatrace-appmon-puppet's Issues

Running recipe dynatrace::server fails to start the server and results in exit code 102

As reported by the server logs:

[03e448d9] info    [native] Process created successfully, pid: 533
[02bf18d9] info    [java  ] ServerHelperAgent loaded
[02bf18d9] info    [java  ] 
[03e448d9] info    [native] Process 533 terminated, exitcode: 102
[03e448d9] severe  [native] Process terminated unexpectedly!

Integrate checking of md5 sum for public download links

A simple approach could look as follows:

After downloading ${component_installer_file_url}, try to download ${component_installer_file_url}.md5 to the cache directory. Example: http://downloads.dynatracesaas.com/6.2/dynatrace-collector-linux-x86.jar.md5 after http://downloads.dynatracesaas.com/6.2/dynatrace-collector-linux-x86.jar.
In the component's install script, check if an .md5 file is in place next to the installer artifact and verify using md5sum. Then, exit 1 if the sum could be calculated, but does not match and exit 0 otherwise. Example: templates/collector/install-collector.sh.

@nnmathieugrin, @gehel: your feedback on this is highly appreciated.

Agent strings are injected multiple times

The file_line resource in manifests/role/apache_wsagent.pp and manifests/role/java_agent.pp requires a match attribute to be defined.

how do you update path for execute("jar -xf #{installer_path} #{init_script}")

for provider dynatrace_installation, where do you define path for execute("jar -xf #{installer_path} #{init_script}")

puppet install module looks for non existent file

hello
It seems every puppet agent run looks for a file called $wsagent_package_installer_prefix_dir/dynatrace/agent/conf/agentversion.dtwsagent - it doesn't find it so it puts the init script in place and does a few other things. this is on every single puppet run. however the file doesn't exist in the v7 agent tarball on linux. I can't seem to find a param to override so it won't look for this file:

`Info: Applying configuration version '9c7770cc80a6f909c550ee82a799d91e986b0798'
Checking if requires installation for path: /dynatrace-web/dynatrace/agent/conf/agentversion.dtwsagent. Result: true, ensure= present
script2exec=/opt/puppetlabs/server/data/puppetserver/dynatrace/install-wsagent-package.sh
Notice: /Stage[main]/Dynatraceappmon::Role::Wsagent_package/Dynatrace_installation[Install the Dynatrace WebServer Agent]/ensure: created
Notice: /Stage[main]/Dynatraceappmon::Role::Wsagent_package/Dynatraceappmon::Resource::Configure_init_script[dynaTraceWebServerAgent]/File[Configure and copy the Dynatrace WebServer Agent's 'dynaTraceWebServerAgent' init script]/content:
--- /dynatrace-web/dynatrace/init.d/dynaTraceWebServerAgent 2017-05-30 13:01:01.000000000 +0100
+++ /tmp/puppet-file20180105-30709-bzbnqi 2018-01-05 14:34:19.938745416 +0000
@@ -22,18 +22,18 @@

- DT_OPTARGS optional arguments (server, ...)

- DT_RUNASUSER the user to run the Dynatrace process

-DT_HOME="/opt/dynatrace-7.0"
`

If you run "puppet agent" from /etc, /etc/init.d is removed (critical!)

It seems that if you run puppet agent from /etc, the /etc/init.d symlink is removed.

Problem:

lib/puppet/provider/dynatrace_installer/jar.rb

This fix somehow solves the issue, but is not really nice either:

  def get_install_dir(installer_path)

    FileUtils.cd("/tmp")

    # extract an init script (includes reference to the dynatrace-x.y.z dir)
    init_script = execute("jar -tf #{installer_path} | grep -e 'init.d' | tail -n 1").strip
    execute("jar -xf #{installer_path} #{init_script}")

    # extract the dynatrace-x.y.z directory name from the init script
    install_dir = execute("grep -e 'DT_HOME=' #{init_script} | cut -d'=' -f2 | xargs basename").strip

    # remove temporary artefacts
    FileUtils.rm_rf('init.d')

    return install_dir
  end

Create cache directory as array of directories

See https://projects.puppetlabs.com/issues/86#note-40.

The dynatrace installation code gets executed each Puppet run.

I'm having trouble finding out what is causing this. Puppet run debug output shows the following for each and every Puppet run:

Debug: Exec[Check for the presence of /var/opt/lib/pe-puppet/dynatrace/dynatrace.jar](provider=posix): Executing check '/usr/bin/test -e /var/opt/lib/pe-puppet/dynatrace/dynatrace.jar' Debug: Executing '/usr/bin/test -e /var/opt/lib/pe-puppet/dynatrace/dynatrace.jar' Debug: Exec[wget-Download http://<our server>/various/dynatrace/6.3/dynatrace-linux-x64.jar to /var/opt/lib/pe-puppet/dynatrace/dynatrace.jar](provider=posix): Executing check 'test -s '/var/opt/lib/pe-puppet/dynatrace/dynatrace.jar'' Debug: Executing 'test -s '/var/opt/lib/pe-puppet/dynatrace/dynatrace.jar'' Debug: Executing 'jar -tf /var/opt/lib/pe-puppet/dynatrace/dynatrace.jar | grep -e 'init.d' | tail -n 1' Debug: Executing 'jar -xf /var/opt/lib/pe-puppet/dynatrace/dynatrace.jar init.d/dynaTracezRemoteAgent' Debug: Executing 'grep -e 'DT_HOME=' init.d/dynaTracezRemoteAgent | cut -d'=' -f2 | xargs basename' Notice: /Stage[main]/Dynatrace::Role::Server/Dynatrace_installation[Install the Dynatrace Server]/ensure: created Debug: /Stage[main]/Dynatrace::Role::Server/Dynatrace_installation[Install the Dynatrace Server]: The container Class[Dynatrace::Role::Server] will propagate my refresh event Debug: Executing '/sbin/service dynaTraceServer status' Debug: Executing '/sbin/chkconfig dynaTraceServer' Debug: Class[Dynatrace::Role::Server]: The container Stage[main] will propagate my refresh event Debug: Executing '/sbin/service dynaTraceCollector status' Debug: Executing '/sbin/chkconfig dynaTraceCollector' Debug: Executing '/sbin/service dynaTraceAnalysis status' Debug: Executing '/sbin/chkconfig dynaTraceAnalysis'

This is on RedHat 6.7 running PE 3.7.2

Puppet module requires Ruby 2.2, should be migrated to run on Ruby 1.9

As Ruby 1.9 is more widely distributed.

puppetlabs/java - jre install

It would be useful to have a fallback when we only have a jre install
the jre has no more the jar binary
would it be possible to have something like yes | java -jar installer ?

Publish module on Puppet Forge

Depends on #3.

init.d file for WebServerAgent in v1.0.9 is different from that of inmstallation script in 7.0

Hi,

For installing the WebServerAgents in AppMon 7.0, we use Dynatrace-AppMon-Puppet v1.0.9.
Unfortunately, the installation script "dynatrace-wsagent-7.0.0.2469-linux-x64.sh" coming from the "dynatrace-wsagent.tar" package of 7.0 is different from the template defined in "Dynatrace-AppMon-Puppet v1.0.9", especially starting on line 52.

During the run of the Puppet agent, it generates ALWAYS a notice about the change of the md5:
"Notice: /Stage[main]/Dynatraceappmon::Role::Wsagent_package/ Dynatraceappmon::Resource::Configure_init_script[dynaTraceWebServerAgent]/File[Configure and copy the Dynatrace WebServer Agent's 'dynaTraceWebServerAgent' init script]/content: content changed '{md5}b8494edb9699167b698baa4a901d10fb' to '{md5}28a688cd78b2d2ec89648e1fac861350'".

Since it will probably not be easy to modfiy the tar package, could it be possible to adapt the init.d file in the puppet module in order to avoid getting this notice?

Thanks in advance.

every single puppet agent run the http.conf is populated and httpd is restarted

every single puppet agent run the http.conf is populated and httpd is restarted:

Notice: /Stage[main]/Apache/File[/etc/httpd/conf/httpd.conf]/content:
--- /etc/httpd/conf/httpd.conf 2018-01-05 15:32:41.950606029 +0000
+++ /tmp/puppet-file20180105-5498-1lhcgwv 2018-01-05 15:34:12.892985003 +0000
@@ -47,4 +47,3 @@

IncludeOptional "/etc/httpd/conf.d/*.conf"

-LoadModule dtagent_module "/dynatrace-web/dynatrace/agent/lib64/libdtagent.so"

Notice: /Stage[main]/Apache/File[/etc/httpd/conf/httpd.conf]/content:

Notice: /Stage[main]/Apache/File[/etc/httpd/conf/httpd.conf]/content: content changed '{md5}ab4b75b42dd169e089d9826a1e8e3a39' to '{md5}fe72ef5e79329ba4d02c161963dd3ed4'
Info: /Stage[main]/Apache/File[/etc/httpd/conf/httpd.conf]: Scheduling refresh of Class[Apache::Service]
Info: Class[Apache::Service]: Scheduling refresh of Service[httpd]
Notice: /Stage[main]/Apache::Service/Service[httpd]: Triggered 'refresh' from 1 events
Notice: /Stage[main]/Dynatraceappmon::Role::Apache_wsagent/File_line[Inject the Dynatrace Apache WebServer Agent into Apache HTTPD's config file]/ensure: created
Notice: Applied catalog in 4.03 seconds

Server and Collector cannot be installed on the same host

The following bug has been submitted by a user: the class declaration class { 'dynatrace::role::dynatrace_user': }, which exists in both dynatrace::role::collector and dynatrace::role::server makes the Puppet run fail with Error: Duplicate declaration (when they are invoked in the same manifest):

require_relative issue

Hi Martin. Just reporting this issue. All the time I install the modules (either server,collector etc) I got this error. They are related to jar.rb and tar.rb
Any way to fix this? I had to manually change require_relative to require to fix but not sure if it's the right way.

Error: Could not autoload puppet/provider/dynatrace_installer/tar: undefined method require_relative' for main:Object Error: Could not autoload puppet/type/dynatrace_installer: Could not autoload puppet/provider/dynatrace_installer/tar: undefined methodrequire_relative' for main:Object
Error: /Stage[main]/Dynatrace::Role::Server/Dynatrace_installation[Install the Dynatrace Server]: Could not evaluate: Could not autoload puppet/type/dynatrace_installer: Could not autoload

init.d scripts and Dynatrace 7

It seems the init.d scripts provided are for Dynatrace 6.3 (e.g. https://github.com/Dynatrace/Dynatrace-AppMon-Puppet/blob/master/templates/init.d/dynaTraceServer.erb#L45). Is there something Dynatrace 7 compatible?

Facter: error while resolving custom fact "install_server__check_service": No such file or directory - pgrep -f "dtserver"

On every puppet run on a windows 2012r2 server we are getting the error
Facter: error while resolving custom fact "install_server__check_service": No such file or directory - pgrep -f "dtserver"

looking at the code
lib\facter\install_server__check_service.rb
it looks like the issue is we do not have pgrep on puppet agents or windows boxes. For now the quick fix i would recommend is adding
confine :kernel => 'Linux'
to line 2 so it would look something like

Facter.add(:install_server__check_service) do
  confine :kernel => 'Linux'
  setcode do

this will allow the fact to work in Linux and stop the error for windows, ( and i am not sure if dtserver even runs on windows)

Automatically test Performance Warehouse Connectivity

Establishing connectivity with a Performance Warehouse should become part of our automated integration tests using Test Kitchen.

Server download link uses unofficial artifact

In
https://github.com/Dynatrace/Dynatrace-AppMon-Puppet/blob/master/manifests/params.pp

The link is
http://files.dynatrace.com/downloads/OnPrem/dynaTrace/$%7Bdynatrace_version_link%7D/dynatrace-server-$%7Bdynatrace_version%7D-linux-x86.jar%22
However, the dynatrace-server-X file is an internal-only artifact, which is affected by an issue SUPDT-37349.

Please switch to the full installer file as found, eg here:
https://files.dynatrace.com/downloads/OnPrem/dynaTrace/7.0/7.0.0.2469/dynatrace-full-7.0.0.2469-linux-x86-64.jar

FYI, the file is now removed from that download location!

If required, please add an RFE for full supported release of that artifact.

The PID file shouldn't be created in a temporary folder

In the init.d service script, there is a hard coded value for DT_PIDFILE_DIR whichis /tmp.
Writing the pid file to a temporary folder is a dangerous thing, there is no persistency guarantee and a system or administrator could easily delete the PID file.
This would lead to a situation where the service seams dead when it is in fact still running, Puppet would then try to restart it and face port conflict.

Checking if requires installation for path: /opt/dynatrace-6.3/agent. Result: false, ensure= present