Giter VIP home page Giter VIP logo

pipeline-maven-plugin's Introduction

Pipeline Maven Plugin

Build Contributors Jenkins Plugin Installs Plugin GitHub release

Jenkins logo plus Maven Owl logo

The Pipeline Maven Plugin provides an advanced set of features for using Apache Maven in Jenkins Pipelines.

The withMaven step configures a maven environment to use within a pipeline job by calling sh "mvn …​ or bat "mvn …​. The selected maven installation is configured and prepended to the path.

It provides several Features to simplify the creation of your Pipeline scripts and ensure the Traceability of Maven builds.

It uses some Sensible default Maven parameters for a CI usage (batch-mode, …​) and automatically configures the Jenkins Report Publishers (Artifacts, JUnit, …​).

The Default Configuration can be defined globally or at the folder level to keep your pipeline code as short as possible while using the expected configuration (Maven settings.xml file…​) globally or locally.

This plugin allows transitioning smoothly from the legacy Maven Integration job type by allowing to reuse Maven Settings Support and by proposing the Trigger downstream pipeline when a snapshot is built.

Installation

The plugin can be found under the name Pipeline Maven Integration Plugin in the Default Update Site.

For tests purposes only, pre-releases (beta) versions are available in the Experimental Plugin Site and development versions (incremental releases) are archived on every successful build on our CI environment.

🔥
If you use the feature Trigger downstream pipeline when a snapshot is built it is critical to setup a real production database instead of the default embedded H2 instance. If you don’t do it, it might degrade your environment stability (The H2 database is running in your instance JVM and could request too many resources).

Changelog

Release notes are recorded in GitHub Releases.

Usage

The Pipeline Maven Plugin works with Linux, Windows and MacOSX build agents.

Simple Maven build on a Linux agent.
node {
  stage ('Build') {
    git url: 'https://github.com/cyrille-leclerc/multi-module-maven-project'
    withMaven {
      sh "mvn clean verify"
    } // withMaven will discover the generated Maven artifacts, JUnit Surefire & FailSafe reports and FindBugs reports
  }
}

It can be supports various parameters to fine-tune its behavior.

Within a node or a docker.image block, create a withMaven block to set up a with maven environment.

The configured environment will be used when calling maven inside the block by using sh mvn or bat mvn.

The following parameters can be used to configure Maven:

  • Maven (maven): Allow the selection of a Maven installation configured on the Global Jenkins configuration or on the Global Tool Configuration page if using Jenkins > 2.0. When auto-install is enabled, maven will be downloaded and made available for the pipeline job.

  • JDK (jdk): Allows the selection of a JDK installation. If auto-install is enabled, the JDK will be downloaded and made available for the pipeline job.

  • Maven Settings

    • Maven Settings Config (mavenSettingsConfig): Select a Maven settings file ID from Config File Provider Plugin allowing the replacement of server credentials and variable substitutions as configured in Config File Provider Plugin. The settings' element in the settings.xml file contains elements used to define values which configure Maven execution in various ways, like the pom.xml, but should not be bundled to any specific project, or distributed to an audience. See also settings.xml reference

    • Maven Settings File Path (mavenSettingsFilePath): Specify the path to a Maven settings.xml file on the build agent. The specified path can be absolute or relative to the workspace.

    • If none of mavenSettingsConfig and mavenSettingsFilePath are defined, withMaven(){} will use the Maven settings defined in the Jenkins Global Tool Configuration if declared

  • Maven Global Settings

    • Maven Global Settings Config (globalMavenSettingsConfig): Select a Maven global settings file ID from Config File Provider Plugin.

    • Maven Global Settings File Path (globalMavenSettingsFilePath): Specify the path to a Maven global settings.xml file on the build agent. The specified path can be absolute or relative to the workspace.

    • If none of globalMavenSettingsConfig and`globalMavenSettingsFilePath` are defined, withMaven(){} will use the Maven global settings defined in the Jenkins Global Tool Configuration if declared

  • Maven JVM Opts (mavenOpts): Specify JVM specific options needed when launching Maven as an external process, these are not maven specific options. See: Java Options
    Shell-like environment variable expansions work in this field, by using the ${VARIABLE} syntax.

  • Maven Local Repository (mavenLocalRepo): Specify a custom local repository path. Shell-like environment variable expansions work with this field, by using the ${VARIABLE} syntax. Normally, Jenkins uses the local Maven repository as determined by Maven, by default ~/.m2/repository and can be overridden by <localRepository> in ~/.m2/settings.xml (see Configuring your Local Repository))
    This normally means that all the jobs that are executed on the same node shares a single Maven repository. The upside of this is that you can save the disk space, the downside is that the repository is not multi process safe and having multiple builds run concurrently can corrupt it. Additionally, builds could interfere with each other by sharing incorrect or partially built artifacts. For example, you might end up having builds incorrectly succeed, just because your have all the dependencies in your local repository, despite that fact that none of the repositories in POM might have them.
    By using this option, Jenkins will tell Maven to use a custom path for the build as the local Maven repository by using -Dmaven.repo.local
    If specified as a relative path then this value will be resolved against the workspace root and not the current working directory.
    ie. $WORKSPACE/.repository if .repository value is specified.

mavenSettingsConfig and globalMavenSettingsConfig use the ID, not the name, of the Maven settings file (resp Maven Global Settings file).
💡
The Pipeline Syntax snippet code generator can be used to assist on generating the withMaven step parameters!
More complex Maven build on a Windows agent.
node ("windows") {
  stage ('Build') {

    git url: 'https://github.com/cyrille-leclerc/multi-module-maven-project'

    withMaven(
        // Maven installation declared in the Jenkins "Global Tool Configuration"
        maven: 'maven-3', // (1)
        // Use `$WORKSPACE/.repository` for local repository folder to avoid shared repositories
        mavenLocalRepo: '.repository', // (2)
        // Maven settings.xml file defined with the Jenkins Config File Provider Plugin
        // We recommend to define Maven settings.xml globally at the folder level using
        // navigating to the folder configuration in the section "Pipeline Maven Configuration / Override global Maven configuration"
        // or globally to the entire master navigating to  "Manage Jenkins / Global Tools Configuration"
        mavenSettingsConfig: 'my-maven-settings' // (3)
    ) {

      // Run the maven build
      sh "mvn clean verify"

    } // withMaven will discover the generated Maven artifacts, JUnit Surefire & FailSafe & FindBugs & SpotBugs reports...
  }
}

In the above example the following parameters are used to configure Maven:

  1. maven: 'maven-3' Maven Installation will be used, this installation has to be declared in the Global Jenkins configuration or Tool installations page.

  2. mavenLocalRepo: a local repository folder is specified to avoid shared repositories

  3. mavenSettingsConfig: specifies a specific settings.xml configuration from Config File Provider Plugin, allowing the replacement of variables and credentials.

Features

Sensible default Maven parameters

The Maven parameters that are useful on a build server, --batch-mode(-B) and --show-version (-V) are enable by default, no need to add them in your mvn invocations.

Maven Settings Support

The withMaven() pipeline step will setup the Maven settings file and global settings file either explicitly using the attributes of the withMaven(){} step declaration or implicitly using the Maven Global Settings and Settings files defined at the folder level or in the Jenkins Global Tools Configuration.

Using implicit declaration, Jenkins administrators can simplify the work of pipeline authors hiding the "boilerplate" to declare the credentials of the Git, Nexus, Artifactory…​ servers and all the needed proxies, mirrors…​

global tools configuration maven settings default maven settings defined at the folder level

Traceability of Maven builds

The withMaven() pipeline step will capture in the logs of the build all the details of the execution:

  • Version of the JVM

    • withMaven(){} step initialization:

[withMaven] use JDK installation JDK8
  • mvn executable invocation:

Java version: 1.8.0_102, vendor: Oracle Corporation
  • Version of Maven

    • withMaven(){} step initialization:

[withMaven] use Maven installation 'M3'
  • mvn executable invocation:

Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00)
  • Name or path of the Maven settings.xml and Maven global settings.xml file.

    • withMaven(){} step initialization:

[withMaven] use Maven settings provided by the Jenkins Managed Configuration File 'maven-settings-for-supply-chain-build-job'
  • When using the Maven settings.xml and global settings.xml files provided by the Jenkins Config File Provider Plugin, details of the Jenkins credentials injected in the Maven build.

    • withMaven(){} step initialization:

[withMaven] use Maven settings.xml 'maven-settings-for-supply-chain-build-job' with Maven servers credentials provided by Jenkins (replaceAll: true): [mavenServerId: 'nexus.beescloud.com', jenkinsCredentials: 'beescloud-nexus-deployment-credentials', username: 'deployment', ...]
Sample
 [withMaven] use JDK installation JDK8
 [withMaven] use Maven installation 'M3'
 [withMaven] use Maven settings provided by the Jenkins Managed Configuration File 'maven-settings-for-supply-chain-build-job'
 [withMaven] use Maven settings.xml 'maven-settings-for-supply-chain-build-job' with Maven servers credentials provided by Jenkins (replaceAll: true):
      [mavenServerId: 'nexus.beescloud.com', jenkinsCredentials: 'beescloud-nexus-deployment-credentials', username: 'deployment', type: 'UsernamePasswordCredentialsImpl'],
      [mavenServerId: 'github.beescloud.com', jenkinsCredentials: 'github-enterprise-api-token', username: 'dev1', type: 'UsernamePasswordCredentialsImpl']
 ...
 Running shell script
 + mvn clean deploy
 ----- withMaven Wrapper script -----
 Picked up JAVA_TOOL_OPTIONS: -Dmaven.ext.class.path=".../pipeline-maven-spy.jar" -Dorg.jenkinsci.plugins.pipeline.maven.reportsFolder="..."
 Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00)
 Maven home: /home/ubuntu/jenkins-home/tools/hudson.tasks.Maven_MavenInstallation/M3
 Java version: 1.8.0_102, vendor: Oracle Corporation
 Java home: /home/ubuntu/jenkins-home/tools/hudson.model.JDK/JDK8/jre
 Default locale: en_US, platform encoding: UTF-8
 OS name: "linux", version: "3.13.0-109-generic", arch: "amd64", family: "unix"

Report Publishers

Maven build executions inside the withMaven(){} will be detected and Jenkins will transparently

  • Archive and fingerprint generated Maven artifacts and Maven attached artifacts

  • Publish JUnit / Surefire reports (if the Jenkins JUnit Plugin is installed)

  • Concordion test reports (since 3.0.0)

  • And more (see below) …​

⚠️
Deprecated publishers
💡
In the future, deprecated publishers should be replaced by Warnings Next Generation implementations (See: JENKINS-57427)
ℹ️
The detection of Maven builds requires using Maven 3.2+.
Reporter Description Required Jenkins Plugin (1) Configuration to disable the feature
Since v2.3.0
Marker file to disable the feature (2)

Generated Artifact

Archiving and the fingerprinting of the artifacts and attached artifacts generated by the Maven build (jar, sources jar, javadocs jar…​)

withMaven(options: [artifactsPublisher(disabled: true)],…​)

.skip-archive-generated-artifacts

Generated JUnit reports

Publishing of the JUnit reports generated from the Surefire, FailSafe, Tycho, Karma or Frontend plugins during the Maven build. Except the Frontend one, all these plugins publish a reportsDirectory property which can be used to find and import JUnit compatible reports. The Frontend plugin has another behaviour, so for detection to work, you have to set a REPORTS_DIRECTORY environment variable to the plugin (and reuse it in your Karma configuration, to be consistent) :

   <plugin>
        <groupId>com.github.eirslett</groupId>
        <artifactId>frontend-maven-plugin</artifactId>
        <executions>
            <execution>
                ...
                <configuration>
                    ...
                    <environmentVariables>
                        <REPORTS_DIRECTORY>${project.build.directory}/karma-reports</REPORTS_DIRECTORY>
                    </environmentVariables>
                </configuration>
            </execution>
        </executions>
    </plugin>

withMaven(options: [junitPublisher(disabled: true)],…​)

.skip-publish-junit-results

Generated Findbugs reports

Publishing of the Findbugs reports generated by the Maven build

withMaven(options: [findbugsPublisher(disabled: true)],…​)

.skip-publish-findbugs-results

Tasks scanner report

Publishing of a report of the “FIXME” and “TODO” tasks found in the java source code. The keywords can be configured.

withMaven(options: [openTasksPublisher(disabled: true)],…​)

.skip-task-scanner

Dependencies Fingerprinting
(since 2.5.0)

Fingerprint the Maven dependencies.
By default, only the snapshot dependencies of scope compile, runtime and provided are fingerprinted.

withMaven(options: [dependenciesFingerprintPublisher(disabled: true)],…​)

.skip-fingerprint-maven-dependencies

Concordion test report
(since 3.0.0)

Publishing of the Concordion test reports.
Publish the Concordion reports generated by the maven-surefire-plugin:test and the `maven-failsafe-plugin:integration-test goals and located in the folder described by the system property `concordion.output.dir as documented in Concordion > Integration > Java > Maven

withMaven(options: [concordionPublisher(disabled: true)],…​)`

.skip-publish-concordion-results

Maven Invoker Plugin test reports

Publish test reports generated by the maven-invoker-plugin:run goal

Maven Invoker Plugin

withMaven(options: [invokerPublisher(disabled: true)],…​)

.skip-publish-invoker-run

JGiven reports

Publish JGiven test reports

withMaven(options: [jgivenPublisher(disabled: true)],…​)

.skip-publish-jgiven-results

JaCoCo Code Coverage

Publish JaCoCo Code Coverage

withMaven(options: [jacocoPublisher(disabled: true)],…​)

Maven Linker Publisher

Publish the Maven report on the pipeline build GUI (list of dependencies, produced artifacts, downstream & upstream pipelines).
This publisher should be renamed "Maven Build Report Publisher".
A reason to disable this publisher is typically to not "pollute" the build screen with Maven invocations when Maven is used as a utility (e.g. invocations of "maven-help-plugin:3.2.0:evaluate"…​)

withMaven(options: [mavenLinkerPublisher(disabled: true)],…​)

skip-maven-linker-publisher

Pipeline Graph Publisher

Build the graph of dependencies between Jenkins pipelines and Maven artifacts in order to trigger downstream pipelines (when using the snapshotDependencies on downstream pipelines)

withMaven(options: [pipelineGraphPublisher(disabled: true)],…​)

.skip-pipeline-graph

(1) Jenkins Plugin to publish the reports on the Jenkins build page. If the plugin is not installed, then the Maven report is ignored.

(2) Marker file to temporarily disable the feature for a specific Maven build. Typically, used to disable a reporter for a specific build that would generate too much data for the default configuration of the reporter (e.g. too many generated artifacts…​) or to workaround a bug in the “withMaven” waiting for a fix. These markers file must be located in the home directory of the build.

Implicit or Explicit activation of Publishers

By default, all the publishers are enabled by default.

It is possible to change the default activation of a publisher navigating to the Global Tool Configuration screen.

It is possible to disable the default activation of publishers on a specific withMaven(){…​} step using the publisherStrategy='EXPLICIT' attribute in the step withMaven(publisherStrategy='EXPLICIT'){…​}. The publishers can then be enabled explicitly in the withMaven(){…​} step using the "publishers" attribute.

Default Configuration

Default Maven settings can be defined globally and at the folder level.

Global Default Configuration

In the Global Tool Configuration screen

  • Maven settings and Maven global settings

  • Publishers settings: enable/disable publishers…​

pipeline maven plugin global tools configuration

Folder Level Configuration

In the Folder configuration screen

  • Maven settings and Maven global settings

pipeline maven folder level configuration

Trigger downstream pipeline when a snapshot is built

ℹ️
Available since version 3.0.0

Trigger downstream pipeline that depend on Maven artifact generated by upstream pipelines.

ℹ️
  • The upstream artifact must be generated in a withMaven(){} wrapping step to be detected by the triggering system

  • The downstream pipeline must have selected the build trigger Build whenever a SNAPSHOT dependency is built

    • The build trigger can be defined at the pipeline level (Build Triggers), at the multibranch pipeline level (Scan Repository Triggers) or at the GitHub Organization / Bitbucket Project level (Scan Organizations Triggers)

  • You have to manually trigger once the upstream pipeline, and the downstream pipeline so that the link between the pipelines based on the SNAPSHOT dependency is established

  • The dependency graph is, for the moment, exclusively stored in an H2 embedded database ($JENKINS_HOME/jenkins-jobs/jenkins-jobs.mv.db). Support for an external H2 database and then for alternate databases (PostgreSQL) is on the roadmap (see PipelineMavenPluginH2Dao.java)

trigger downstream 1 trigger downstream 2

Downstream Pipeline Trigger - Org Level Configuration

  • Thresholds are applied to define on which type of maven build the downstream pipelines are triggered

    • Threshold based on the status of the upstream pipeline (success, unstable, failure, no build, aborted). By default, only builds with a success result will trigger downstream builds. downstream pipeline trigger threshold build result

    • Threshold based on the Maven lifecycle phase reached in the Maven build of the upstream job (package, install, deploy). By default, only the maven builds who reach the deploy phase will trigger downstream builds. downstream pipeline trigger threshold lifecycle

Support of Takari’s Maven Wrapper 'mvnw'

ℹ️
Available since version 3.0.3

The Pipeline Maven Plugin works with Takari’s Maven wrapper 'mvnw'.

withMaven() {
   sh "./mvnw clean deploy"
}

Adding more Maven Reporters

The API for Maven reporters is still experimental. Please open a Request for Enhancement Jira issue to discuss how to add Maven reporters.

We want to quickly add reporters for CheckStyle, Jacoco…​

Database Setup

Using a MySQL Database

ℹ️
Available since version 3.6.0

The Jenkins Pipeline Maven Plugin relies on a database to store its data (list of dependencies and of generated artifacts of each build…​).

By default, the Jenkins Pipeline Maven Plugin uses an H2 embedded database, but it is recommend to use an external MySQL database.

Configuration steps to use a MySQL:

  • Create an empty MySQL database with a dedicated MySQL user with permissions for Data Manipulation Language actions (DML) and Data Definition Language (DDL) actions

    • Tested with MySQL 8.0, with MariaDB 10.2 and 10.3 and with Amazon Aurora MySQL 5.6

  • Install the Jenkins MySQL Database plugin

    • Navigate to Manage Jenkins / Manage Plugins / Available, select the MySQL Database plugin and click on Download now and install after restart

  • Configure the Pipeline Maven Plugin to use the created MySQL database

    • Create Jenkins credentials for the MySQL connection navigating to Credentials on the left menu

    • Navigate to Manage Jenkins / Global Tools Configuration and go to the Pipeline Maven Configuration pipeline maven plugin configuration 1

    • In the Database configuration section, define the following.

      • JDBC URL: url of the database, e.g. jdbc:mysql://mysql.example.com/jenkins

      • JDBC Credentials: select the credentials of the MySQL database.

      • The parameters recommended by the Hikari Connection Pool team are used by default for the MySQL connections and for the datasource (max pool size:10, server side prepared statements cache with 250 entries…​). To overwrite these defaults, click on the Advanced Database Configuration button.

    • Click on Validate Database Configuration button to verify that the connection is successful.

    • Click on Save. pipeline maven plugin configuration 2

Using a PostgreSQL Database

ℹ️
Available since version 3.7.0

The Jenkins Pipeline Maven Plugin relies on a database to store its data (list of dependencies and of generated artifacts of each build…​).

By default, the Jenkins Pipeline Maven Plugin uses an H2 embedded database, but it is recommended to use an external PostgreSQL or MySQL / MariaDB database.

Configuration steps to use a PostgreSQL:

  • Create an empty PostgreSQL database with a dedicated PostgreSQL user with permissions for Data Manipulation Language actions (DML) and Data Definition Language (DDL) actions

    • Tested with PostgreSQL 10.6 and 11.3

  • Install the Jenkins the PostgreSQL API plugin

    • Navigate to Manage Jenkins / Manage Plugins / Available, select the PostgreSQL API plugin and click on Download now and install after restart.

  • Configure the Pipeline Maven Plugin to use the created PostgreSQL database

    • Create Jenkins credentials for the PostgreSQL connection navigating to Credentials on the left menu

    • Navigate to Manage Jenkins / Global Tools Configuration and go to the Pipeline Maven Configuration pipeline maven plugin configuration postgresql

    • In the Database configuration section, define the following

      • JDBC URL: url of the database, e.g. `jdbc:postgresql://postgresql.example.com:5432/jenkins

      • JDBC Credentials: select the credentials of the PostgreSQL database

      • The underlying datasource, HikariCP, comes with sensible default configuration values (see here). To overwrite these defaults, click on the Advanced Database Configuration button.

    • Click on Validate Database Configuration button to verify that the connection is successful.

    • Click on Save.

    • Navigate to Manage Jenkins / Global Tools Configuration and go to the Pipeline Maven Configuration to verify that the database connection is successful, and the database tables have been created (see screenshot above).

Known Limitations

Maven and JDK installation not supported in docker.image('xxx').inside{…​}

Maven and JDK installers do not work with docker.image('xxx').inside{…​} as the docker step does not allow the use of Tool Installer, the preinstalled Maven and JDK on the docker image will be auto-discovered and used.

withMaven() not supported in docker.image('xxx').inside{…​} with old versions of the Docker engine

withMaven() not supported in docker.image('xxx').inside{…​} with old versions of the Docker engine such as Docker 1.13.1 on CentOS7.

Any help to fix this bug is more than welcome.

JENKINS-40484 - Getting issue details…​ STATUS

Frequently Asked Questions

See FAQ

pipeline-maven-plugin's People

Contributors

aheritier avatar cyrille-leclerc avatar alvarolobato avatar dependabot[bot] avatar jglick avatar imod avatar bingshiao avatar daniel-beck avatar brenuart avatar abayer avatar jtnord avatar jcoste-orange avatar janschultze avatar tneer avatar slachiewicz avatar sgrillon14 avatar pauxus avatar olamy avatar markewaite avatar marioja avatar jlleitschuh avatar jcoeltjen avatar famod avatar cschulz avatar amirschnell avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.