Giter VIP home page Giter VIP logo

jenkinspipelineunit's Introduction

JenkinsPipelineUnit Testing Framework

Jenkins Pipeline Unit is a testing framework for unit testing Jenkins pipelines, written in Groovy Pipeline DSL.

Linux/Windows Build status Mac Build status GitHub release (latest SemVer) Gitter chat

If you use Jenkins as your CI workhorse (like us @ lesfurets.com) and you enjoy writing pipeline-as-code, you already know that pipeline code is very powerful but can get pretty complex.

This testing framework lets you write unit tests on the configuration and conditional logic of the pipeline code, by providing a mock execution of the pipeline. You can mock built-in Jenkins commands, job configurations, see the stacktrace of the whole execution and even track regressions.

Table of Contents

  1. Usage
  2. Configuration
  3. Declarative Pipeline
  4. Testing Shared Libraries
  5. Writing Testable Libraries
  6. Note On CPS
  7. Contributing
  8. Demos and Examples

Usage

Add to Your Project as Test Dependency

JenkinsPipelineUnit requires Java 11, since this is also the minimum version required by Jenkins. Also note that JenkinsPipelineUnit is not currently compatible with Groovy 4, please see this issue for more details.

Note: Starting from version 1.2, artifacts are published to https://repo.jenkins-ci.org/releases.

Maven

<repositories>
    <repository>
    <id>jenkins-ci-releases</id>
    <url>https://repo.jenkins-ci.org/releases/</url>
    </repository>
    ...
</repositories>

<dependencies>
    <dependency>
        <groupId>com.lesfurets</groupId>
        <artifactId>jenkins-pipeline-unit</artifactId>
        <version>1.9</version>
        <scope>test</scope>
    </dependency>
    ...
</dependencies>

Gradle

repositories {
    maven { url 'https://repo.jenkins-ci.org/releases/' }
    ...
}

dependencies {
    testImplementation "com.lesfurets:jenkins-pipeline-unit:1.9"
    ...
}

Start Writing Tests

You can write your tests in Groovy or Java, using the test framework you prefer. The easiest entry point is extending the abstract class BasePipelineTest, which initializes the framework with JUnit.

Let's say you wrote this awesome pipeline script, which builds and tests your project:

def execute() {
    node() {
        String utils = load 'src/test/jenkins/lib/utils.jenkins'
        String revision = stage('Checkout') {
            checkout scm
            return utils.currentRevision()
        }
        gitlabBuilds(builds: ['build', 'test']) {
            stage('build') {
                gitlabCommitStatus('build') {
                    sh "mvn clean package -DskipTests -DgitRevision=$revision"
                }
            }

            stage('test') {
                gitlabCommitStatus('test') {
                    sh "mvn verify -DgitRevision=$revision"
                }
            }
        }
    }
}

return this

Now using the Jenkins Pipeline Unit you can write a unit test to see if it does the job:

import com.lesfurets.jenkins.unit.BasePipelineTest

class TestExampleJob extends BasePipelineTest {
    @Test
    void shouldExecuteWithoutErrors() {
        loadScript('job/exampleJob.jenkins').execute()
        printCallStack()
    }
}

This test will print the call stack of the execution, which should look like so:

   exampleJob.run()
   exampleJob.execute()
      exampleJob.node(groovy.lang.Closure)
         exampleJob.load(src/test/jenkins/lib/utils.jenkins)
            utils.run()
         exampleJob.stage(Checkout, groovy.lang.Closure)
            exampleJob.checkout({$class=GitSCM, branches=[{name=feature_test}], extensions=[], userRemoteConfigs=[{credentialsId=gitlab_git_ssh, url=github.com/lesfurets/JenkinsPipelineUnit.git}]})
            utils.currentRevision()
               utils.sh({returnStdout=true, script=git rev-parse HEAD})
         exampleJob.gitlabBuilds({builds=[build, test]}, groovy.lang.Closure)
            exampleJob.stage(build, groovy.lang.Closure)
               exampleJob.gitlabCommitStatus(build, groovy.lang.Closure)
                  exampleJob.sh(mvn clean package -DskipTests -DgitRevision=bcc19744)
            exampleJob.stage(test, groovy.lang.Closure)
               exampleJob.gitlabCommitStatus(test, groovy.lang.Closure)
                  exampleJob.sh(mvn verify -DgitRevision=bcc19744)

Mocking Jenkins Variables

You can define both environment variables and job execution parameters.

import com.lesfurets.jenkins.unit.BasePipelineTest

class TestExampleJob extends BasePipelineTest {
    @Override
    @BeforeEach
    void setUp() {
        super.setUp()
        // Assigns false to a job parameter ENABLE_TEST_STAGE
        addParam('ENABLE_TEST_STAGE', 'false')
        // Defines the previous execution status
        binding.getVariable('currentBuild').previousBuild = [result: 'UNSTABLE']
    }

    @Test
    void verifyParam() {
        assertEquals('false', binding.getVariable('params')['ENABLE_TEST_STAGE'])
    }
}

After calling super.setUp(), the test helper instance is available, as well as many helper methods. The test helper already provides basic variables such as a very simple currentBuild definition. You can redefine them as you wish.

Note that super.setUp() must be called prior to using most features. This is commonly done using your own setUp method, decorated with @Override and @BeforeEach.

Parameters added via addParam are immutable, which reflects the same behavior in Jenkins. Attempting to modify the params map in the binding will result in an error.

Mocking Jenkins Commands

You can register interceptors to mock pipeline methods, including Jenkins commands, which may or may not return a result.

import com.lesfurets.jenkins.unit.BasePipelineTest

class TestExampleJob extends BasePipelineTest {
    @Override
    @BeforeEach
    void setUp() {
        super.setUp()
        helper.registerAllowedMethod('sh', [Map]) { args -> return 'bcc19744' }
        helper.registerAllowedMethod('timeout', [Map, Closure], null)
        helper.registerAllowedMethod('timestamps', []) { println 'Printing timestamp' }
        helper.registerAllowedMethod('myMethod', [String, int]) { String s, int i ->
            println "Executing myMethod mock with args: '${s}', '${i}'"
        }
    }
}

The test helper already includes mocks for all base pipeline steps as well as a steps from a few widely-used plugins. You need to register allowed methods if you want to override these mocks and add others. Note that you need to provide a method signature and a callback (closure or lambda) in order to allow a method. Any method call which is not recognized will throw an exception.

Please refer to the BasePipelineTest class for the list of currently supported mocks.

Some tricky methods such as load and parallel are implemented directly in the helper. If you want to override those, make sure that you extend the PipelineTestHelper class.

Mocking readFile and fileExists

The readFile and fileExists steps can be mocked to return a specific result for a given file name. This can be useful for testing pipelines for which file operations can influence subsequent steps. An example of such a testing scenario follows:

// Jenkinsfile
node {
    stage('Process output') {
        if (fileExists('output') && readFile('output') == 'FAILED!!!') {
            currentBuild.result = 'FAILURE'
            error 'Build failed'
        }
    }
}
@Test
void exampleReadFileTest() {
    helper.addFileExistsMock('output', true)
    helper.addReadFileMock('output', 'FAILED!!!')

    runScript('Jenkinsfile')

    assertJobStatusFailure()
}

Mocking Shell Steps

The shell steps (sh, bat, etc) are used by many pipelines for a variety of tasks. They can be mocked to either (a) statically return:

  • A string for standard output
  • A return code

Or (b), to execute a closure that returns a Map (with stdout and exitValue entries). The closure will be executed when the shell is called, allowing for dynamic behavior.

Here is a sample pipeline and corresponding unit tests for each of these variants.

// Jenkinsfile
node {
    stage('Mock build') {
        String systemType = sh(returnStdout: true, script: 'uname')
        if (systemType == 'Debian') {
            sh './build.sh --release'
            int status = sh(returnStatus: true, script: './test.sh')
            if (status > 0) {
                currentBuild.result = 'UNSTABLE'
            } else {
                def result = sh(
                    returnStdout: true,
                    script: './processTestResults.sh --platform debian',
                )
                if (!result.endsWith('SUCCESS')) {
                    currentBuild.result = 'FAILURE'
                    error 'Build failed!'
                }
            }
        }
    }
}
@Test
void debianBuildSuccess() {
    helper.addShMock('uname', 'Debian', 0)
    helper.addShMock('./build.sh --release', '', 0)
    helper.addShMock('./test.sh', '', 0)
    // Have the sh mock execute the closure when the corresponding script is run:
    helper.addShMock('./processTestResults.sh --platform debian') { script ->
        // Do something "dynamically" first...
        return [stdout: "Executing ${script}: SUCCESS", exitValue: 0]
    }

    runScript("Jenkinsfile")

    assertJobStatusSuccess()
}

@Test
void debianBuildUnstable() {
    helper.addShMock('uname', 'Debian', 0)
    helper.addShMock('./build.sh --release', '', 0)
    helper.addShMock('./test.sh', '', 1)

    runScript('Jenkinsfile')

    assertJobStatusUnstable()
}

Note that in all cases, the script executed by sh must exactly match the string passed to helper.addShMock, including the script arguments, whitespace, etc. For more flexible matching, you can use a pattern (regular expression) and even capture groups:

helper.addShMock(~/.\/build.sh\s--(.*)/) { String script, String arg ->
    assert (arg == 'debug') || (arg == 'release')
    return [stdout: '', exitValue: 2]
}

Also, mocks are stacked, so if two mocks match a call, the last one wins. Combined with a match-everything mock, you can tighten your tests a bit:

@BeforeEach
void setUp() {
    super.setUp()
    helper = new PipelineTestHelper()
    // Basic `sh` mock setup:
    // - generate an error on unexpected calls
    // - ignore any echo (debug) outputs, they are not relevant
    // - all further shell mocks are configured in the test
    helper.addShMock(null) { throw new Exception('Unexpected sh call') }
    helper.addShMock(~/echo\s.*/, '', 0)
}

Analyzing the Mock Execution

The helper registers every method call to provide a stacktrace of the mock execution.

@Test
void shouldExecuteWithoutErrors() {
    runScript('Jenkinsfile')

    assertJobStatusSuccess()
    assertThat(helper.callStack.findAll { call ->
        call.methodName == 'sh'
    }.any { call ->
        callArgsToString(call).contains('mvn verify')
    }).isTrue()
}

This will also check that mvn verify was called during the job execution.

Checking Pipeline Status

Let's say you have a simple script, and you'd like to check its behavior if a step fails.

// Jenkinsfile
node() {
    git 'some_repo_url'
    sh 'make'
}

You can mock the sh step to just update the pipeline status to FAILURE. To verify that your pipeline is failing, you need to check the status with BasePipelineTest.assertJobStatusFailure().

@Test
void checkBuildStatus() {
    helper.registerAllowedMethod('sh', [String]) { cmd ->
        if (cmd == 'make') {
            binding.getVariable('currentBuild').result = 'FAILURE'
        }
    }

    runScript('Jenkinsfile')

    assertJobStatusFailure()
}

Checking Pipeline Exceptions

Sometimes it is useful to verify that a specific exception was thrown during the pipeline run. JUnit 4 and 5 have slightly different mechanisms for doing this.

For both examples below, assume that the following pipeline is being tested:

To do so you can use org.junit.rules.ExpectedException

// Jenkinsfile
node {
    throw new IllegalArgumentException('oh no!')
}

JUnit 4

class TestCase extends BasePipelineTest {
    @Test(expected = IllegalArgumentException)
    void verifyException() {
        runScript('Jenkinsfile')
    }
}

JUnit 5

import static org.junit.jupiter.api.Assertions.assertThrows

class TestCase extends BasePipelineTest {
    @Test
    void verifyException() {
        assertThrows(IllegalArgumentException) { runScript('Jenkinsfile') }
    }
}

Compare the Callstack with a Previous Implementation

One other use of the callstacks is to check your pipeline executions for possible regressions. You have a dedicated method you can call if you extend BaseRegressionTest:

@Test
void testPipelineNonRegression() {
    loadScript('job/exampleJob.jenkins').execute()
    super.testNonRegression('example')
}

This will compare the current callstack of the job to the one you have in a text callstack reference file. To update this file with new callstack, just set this JVM argument when running your tests: -Dpipeline.stack.write=true. You then can go ahead and commit this change in your SCM to check in the change.

Preserve Original Callstack Argument References

The default behavior of the callstack capture is to clone each call's arguments to preserve their values at time of the call should those arguments mutate downstream. That is a good guard when your scripts are passing ordinary mutable variables as arguments.

However, argument types that are not Cloneable are captured as String values. Most of the time this is a perfect fallback. But for some complex types, or for types that don't implement toString(), it can be tricky or impossible to validate the call values in a test.

Take the following simple example:

Map pretendArgsFromFarUpstream = [
    foo: 'bar',
    foo2: 'more bar please',
    aNestedMap: [aa: 1, bb: 2],
    plusAList: [1, 2, 3, 4],
].asImmutable()

node() {
    doSomethingWithThis(pretendArgsFromFarUpstream)
}

pretendArgsFromFarUpstream is an immutable map and will be recorded as a String in the callstack. Your test may want to perform fine-grained validations via map key referencing instead of pattern matching or similar parsing. For example:

assertEquals(2, arg.aNestedMap.bb)

You may want to perform this kind of validation, particularly if your pipelines pass final and/or immutable variables as arguments. You can retain the direct reference to the variable in the callstack by setting this switch in your test setup:

helper.cloneArgsOnMethodCallRegistration = false

Running Inline Scripts

In case you want to have some script executed directly within a test case rather than creating a resource file for it, loadInlineScript and runInlineScript can be used.

@Test
void testSomeScript() {
    Object script = loadInlineScript('''
        node {
            stage('Build') {
                sh 'make'
            }
        }
    ''')

    script.execute()

    printCallStack()
    assertJobStatusSuccess()
}

Note that inline scripts cannot be debugged via breakpoints as there is no file to attach to!

Configuration

The abstract class BasePipelineTest configures the helper with useful conventions:

  • It looks for pipeline scripts in your project in root (./.) and src/main/jenkins paths.
  • Jenkins pipelines let you load other scripts from a parent script with load command. However load takes the full path relative to the project root. The test helper mock successfully the load command to load the scripts. To make relative paths work, you need to configure the path of the project where your pipeline scripts are, which defaults to ..
  • Pipeline script extension, which defaults to jenkins (matches any *.jenkins file)

Overriding these default values is easy:

class TestExampleJob extends BasePipelineTest {
    @Override
    @BeforeEach
    void setUp() {
        baseScriptRoot = 'jenkinsJobs'
        scriptRoots += 'src/main/groovy'
        scriptExtension = 'pipeline'
        super.setUp()
    }
}

This will work fine for such a project structure:

 jenkinsJobs
 โ””โ”€โ”€ src
     โ”œโ”€โ”€ main
     โ”‚ย ย  โ””โ”€โ”€ groovy
     โ”‚ย ย      โ””โ”€โ”€ ExampleJob.pipeline
     โ””โ”€โ”€ test
         โ””โ”€โ”€ groovy
             โ””โ”€โ”€ TestExampleJob.groovy

Declarative Pipelines

To test a declarative pipeline, you'll need to subclass the DeclarativePipelineTest class instead of BasePipelineTest

// Jenkinsfile
pipeline {
    agent none
    stages {
        stage('Example Build') {
            agent { docker 'maven:3-alpine' }
            steps {
                echo 'Hello, Maven'
                sh 'mvn --version'
            }
        }
        stage('Example Test') {
            agent { docker 'openjdk:8-jre' }
            steps {
                echo 'Hello, JDK'
                sh 'java -version'
            }
        }
    }
}
import com.lesfurets.jenkins.unit.declarative.*

class TestExampleDeclarativeJob extends DeclarativePipelineTest {
    @Test
    void shouldExecuteWithoutErrors() {
        runScript("Jenkinsfile")

        assertJobStatusSuccess()
        printCallStack()
    }
}

DeclarativePipelineTest

The DeclarativePipelineTest class extends BasePipelineTest, so you can verify your declarative job the same way as scripted pipelines.

Testing Pipelines That Use Shared Libraries

With Shared Libraries, Jenkins lets you share common code from pipelines across different repositories. Shared libraries are configured in the Jenkins settings and imported with @Library annotation or the library step.

Testing pipeline scripts using external libraries is not trivial because the shared library code is another repository. JenkinsPipelineUnit lets you test shared libraries and pipelines that depend on these libraries.

Here is an example pipeline using a shared library:

@Library('commons')

import net.courtanet.jenkins.Utils

sayHello 'World'

node() {
    stage ('Checkout') {
        def utils = new Utils()
        checkout "${utils.gitTools()}"
    }
    stage ('Build') {
        sh './gradlew build'
    }
    stage ('Post Build') {
        String json = libraryResource 'net/courtanet/jenkins/request.json'
        sh "curl -H 'Content-Type: application/json' -X POST -d '$json' ${acme.url}"
    }
}

This pipeline is using a shared library called commons. Now let's test it:

// You need to import the class first
import static com.lesfurets.jenkins.unit.global.lib.LibraryConfiguration.library

class TestCase extends BasePipelineTest {
    @Test
    void testLibrary() {
        Object library = library()
            .name('commons')
            .retriever(gitSource('[email protected]:libs/commons.git'))
            .targetPath('path/to/clone')
            .defaultVersion("master")
            .allowOverride(true)
            .implicit(false)
            .build()
        helper.registerSharedLibrary(library)

        runScript('job/library/exampleJob.jenkins')

        printCallStack()
    }
}

Notice how the shared library is defined and registered to the helper. The library definition is done via a fluent API which lets you set the same configurations as in Jenkins Global Pipeline Libraries.

The retriever and targetPath fields tell the framework how to fetch the sources of the library, and to which local path. The framework comes with two naive but useful retrievers, gitSource and localSource. You can write your own retriever by implementing the SourceRetriever interface.

Note that properties defaultVersion, allowOverride and implicit are optional with default values master, true and false.

Now if we execute this test, the framework will fetch the sources from the Git repository and load classes, scripts, global variables and resources found in the library. The callstack of this execution will look like the following:

Loading shared library commons with version master
libraryJob.run()
  libraryJob.sayHello(World)
  sayHello.echo(Hello, World.)
  libraryJob.node(groovy.lang.Closure)
     libraryJob.stage(Checkout, groovy.lang.Closure)
        Utils.gitTools()
        libraryJob.checkout({branch=master})
     libraryJob.stage(Build, groovy.lang.Closure)
        libraryJob.sh(./gradlew build)
     libraryJob.stage(Post Build, groovy.lang.Closure)
        libraryJob.libraryResource(net/courtanet/jenkins/request.json)
        libraryJob.sh(curl -H 'Content-Type: application/json' -X POST -d '{"name" : "Ben"}' http://acme.com)

Library Source Retrievers

There are a few types of SourceRetriever implementations in addition to the previously described GitSource one.

ProjectSource Retriever

The ProjectSource retriever is useful if you write tests for the library itself. So it lets you load the library files directly from the project root folder (where the src and vars folders are located).

Then you can use projectSource to point to the location of the library files. Calling projectSource() with no arguments will look for files in the project root. With .defaultVersion('<notNeeded>'), you can load it in pipelines using commons@master or commons@features which would use the same repository.

import static com.lesfurets.jenkins.unit.global.lib.ProjectSource.projectSource

class TestCase extends BasePipelineTest {
    @Override
    @BeforeEach
    void setUp() {
        super.setUp()
        Object library = library()
            .name('commons')
            .defaultVersion('<notNeeded>')
            .allowOverride(true)
            .implicit(true)
            .targetPath('<notNeeded>')
            .retriever(projectSource())
            .build()
        helper.registerSharedLibrary(library)
    }
}

LocalSource Retriever

The LocalSource retriever is useful if you want to verify how well your library integrates with the pipelines. For example you may use pre-copied library files with different versions.

import static com.lesfurets.jenkins.unit.global.lib.LocalSource.localSource

class TestCase extends BasePipelineTest {
    @Override
    @BeforeEach
    void setUp() {
        super.setUp()
        Object library = library()
            .name('commons')
            .defaultVersion('master')
            .allowOverride(true)
            .implicit(false)
            .targetPath('<notNeeded>')
            .retriever(localSource('/var/tmp/'))
            .build()
        helper.registerSharedLibrary(library)
    }
}

In the above example, the retriever would assume that the library files are located at /var/tmp/commons@master.

Loading Libraries Dynamically

There is partial support for loading dynamic libraries. It doesn't implement all the features, but it could be useful sometimes.

Pipeline example:

Object commonsLib = library 'commons'

// Assume that `sayHello` is a singleton in the `commons` library
sayHello 'World'

// Create an instance of a class in the `commons` library
Object utils = net.courtanet.jenkins.Utils.new()

Test class example:

@Test
void testDynamicLibrary() {
    Object library = library()
        .name('commons')
        .retriever(gitSource('[email protected]:libs/commons.git'))
        .targetPath('path/to/clone')
        .defaultVersion('master')
        .allowOverride(true)
        .implicit(false)
        .build()
    helper.registerSharedLibrary(library)
    // Registration for pipeline method 'library' must be made after registering the
    // shared library. Unfortunately, this cannot be moved to the super class.
    helper.registerAllowedMethod('library', [String], { String name ->
        helper.getLibLoader().loadLibrary(name)
        println helper.getLibLoader().libRecords
        return new LibClassLoader(helper, null)
    })

    loadScript('job/library/exampleJob.jenkins')

    printCallStack()
}

Library Global Variables with Library Object Arguments

You might have a library that defines global variables with library class instances as arguments. For example, consider the following library class and global variable:

// src/com/example/Monster.groovy
package com.example

class Monster {
    String moniker

    Monster(String moniker) {
      this.moniker = moniker
    }
}
// vars/monster.groovy
import com.example.Monster

void call(Monster monster) {
    println "${monster.moniker} is always very scary"
}

Your pipeline uses both as follows:

Monster vampire = new Monster('Dracula')
monster(vampire)
// Should print "Dracula is always very scary"

If this does not yield the expected output but instead throws a MissingMethodException with the cause No signature of method: Jenkinsfile.monster() is applicable for argument types: (org.test.Monster) values: [com.example.Monster1@d34db33f] you may need to disable library class preloading in your test setup, which you do with the following switch.

helper.libLoader.preloadLibraryClasses = false

You may need to do this on a test-by-test basis, as disabling class preloading can cause problems in other cases. For example, when you have library classes that require access to the env global.

Writing Testable Libraries

We recommend the following best-practices for organizing pipeline code:

  • Keep complex logic in the Jenkinsfile to a minimum
    • When possible, move complexity to external scripts that the Jenkinsfile executes
    • Move shared functionality to pipeline libraries
    • Likewise, any tricky Groovy logic that can't be easily moved to external scripts should also be placed in pipeline libraries
  • In pipeline libraries, organize logic in classes under src
    • Ideally, JenkinsPipelineUnit is used to test only these classes
  • Use the vars singletons to instantiate classes from src

On External Scripts

In general, it's better to avoid having complex build logic inside of build pipelines. Although tools like JenkinsPipelineUnit are useful in testing pipelines, it's much easier to run build scripts locally (meaning, outside of a Jenkins environment). Languages like Python have much more sophisticated linting and testing tools than Groovy does.

That said, CodeNarc can be used to lint Groovy code, including Jenkinsfile files.

On Pipeline Library Organization

We recommend organizing pipeline libraries such that the bulk of the logic is organized into classes, and the singletons being thin wrappers around these classes. This approach has several advantages:

  • It makes it easier to use OOP practices to organize the code
  • It solves the problem of having to mock singletons inside of other singletons for tests
  • It forces the script context to be injected into the class, which means less mocking of @Library calls and such

Example Pipeline Library Organization

Let's say we have a library responsible for a very complex operation, in this case, adding two numbers together. ๐Ÿ˜„ Here's what that library (let's call it HardMath) might look like:

// src/com/example/HardMath.groovy
package com.example

class HardMath implements Serializable {
  // Jenkinsfile script context, note that all pipeline steps must use this context
  Object script = null

  int complexOperation(int a, int b) {
    // Note the script context is required for `echo`, as it is a pipeline step
    script.echo "Adding ${a} to ${b}"
    return a + b
  }
}
// vars/hardmath.groovy
import com.example.HardMath

int complexOperation(int a, int b) {
  return new HardMath(script: this).complexOperation(a, b)
}
// test/com/example/HardMathTest.groovy
package com.example

import static org.junit.jupiter.api.Assertions.assertEquals

import com.lesfurets.jenkins.unit.BasePipelineTest
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.Test


class HardMathTest extends BasePipelineTest {
  Object script = null

  @Override
  @BeforeEach
  void setUp() {
    super.setUp()
    this.script = loadScript('test/resources/EmptyPipeline.groovy')
  }

  @Test
  void testComplexOperation() {
    int result = new HardMath(script: script).complexOperation(1, 3)
    assertEquals(4, result)
  }
}
// test/resources/EmptyPipeline.groovy
return this

And finally, in some other project's Jenkinsfile:

// Jenkinsfile
@Library('hardmath')

node {
  stage('Hard Math') {
    int result = hardmath.complexOperation(5, 6)
    echo "The result is ${result}"
  }
}

For a larger real-world example of a pipeline library organized like the above and tested with JenkinsPipelineUnit, have a look at python-pipeline-utils.

Note on CPS

If you already fiddled with Jenkins pipeline DSL, you may have experienced strange errors during execution on Jenkins. This is because Jenkins does not directly execute your pipeline in Groovy, but transforms the pipeline code into an intermediate format in order to run Groovy code in Continuation Passing Style (CPS).

The usual errors are partly due to the the sandboxing Jenkins applies for security reasons, and partly due to the serializability Jenkins imposes.

Jenkins requires that at each execution step, the whole script context is serializable, in order to stop and resume the job execution. To simulate this aspect, CPS versions of the helpers transform your scripts into the CPS format and check if at each step your script context is serializable.

To use this experimental feature, you can use the abstract class BasePipelineTestCPS instead of BasePipelineTest. You may see some changes in the callstacks that the helper registers. Note also that the serialization used to test is not the same as what Jenkins uses. You may find some incoherence in that respect.

Contributing

JenkinsPipelineUnit aims to help developers code and test Jenkins pipelines with a shorter development cycle. It addresses some of the requirements traced in JENKINS-33925. If you are willing to contribute please don't hesitate to discuss in issues and open a pull-request.

Demos and Examples

URL Frameworks and Tools Test Subject Test Layers
https://github.com/macg33zr/pipelineUnit Spock, Gradle(Groovy) Jenkinsfile, scripted pipeline, SharedLibrary UnitTest
https://github.com/mkobit/jenkins-pipeline-shared-library-example Spock, Gradle (Kotlin), Junit SharedLibrary Integration, Unit
https://github.com/stchar/pipeline-sharedlib-testharness Junit, Gradle(Groovy) SharedLibrary Integration, Unit
https://github.com/stchar/pipeline-dsl-seed Junit, Spock, Gradle(Groovy) scripted pipeline Integration(jobdsl), Unit
https://github.com/SpencerMalone/JenkinsPipelineIntegration Spock, Gradle(Groovy) SharedLibrary Integration
https://github.com/venosov/jenkins-pipeline-shared-library-example-victor Junit, Gradle(Kotlin) SharedLibrary Unit

jenkinspipelineunit's People

Contributors

alex-vol-sv avatar alex-weatherhead avatar ballab1 avatar bartcarroll avatar brianeray avatar cwiggs avatar dblock avatar dependabot[bot] avatar equincerot avatar iuryalves avatar jsok avatar marcusholl avatar markewaite avatar noble906 avatar notmyfault avatar nre-ableton avatar oleg-nenashev avatar ozangunalp avatar pkwarren avatar pontuslaestadius avatar radekl avatar recuencojones avatar reinholdfuereder avatar seanf avatar stchar avatar svanacker avatar theosotr avatar timja avatar ulricheckhardt avatar wololock avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

jenkinspipelineunit's Issues

Java 7 support

I'm curious why only Java 8 is supported here? Is there anything in particular that prevents you compiling with Java 8 but using Java 7 bytecode so that tests can be written in 7 or 8?

Cannot get loadInterceptor from PipelineTestHelper

I need to add some custom actions before loading script in load step.
It would be if I could do something like that

class TestExample extends BaseRegressionTest {

    @Before
    @Override
    void setUp() {

        super.setUp()
        def loadInterceptor = helper.loadInterceptor 
        def customLoadInterceptor = {String scriptName ->
            //custom actions
           loadInterceptor.setDelegate(delegate)
           loadInterceptor(scriptName) 
        }
        helper.registerAllowedMethod('load', [String.class], customLoadInterceptor)
}

But currently loadInterceptor has protected access visibility. Is it possible to make it public or there is something that requires it to be protected?

Call node() in vars/ raised a groovy.lang.MissingMethodException

Hi guys,

This library is very useful for us for do unit test for shared library. Thanks for your great work. However I meet a same problem with #25 , I known the root cause is I marked vars/ as a source folder. But I have to do this because I need to count the unit test coverage. I just want to know if it's possible to mock node which is called in vars/ ? I can mock some other global variable like currentBuild which is also used in vars/ and works fine. I don't know what's the difference between node and currentBuild.

I'm a new bee for jenkins and groovy, any help is appreciated. Thanks very much.
Here is my source code:

withWrapNode.groovy:

def call(String label, Closure body = null) {

    def config = [:]
    body.resolveStrategy = Closure.DELEGATE_FIRST
    body.delegate = config
    body()

    node {
        echo "hello test message: ${config.message}"
    }
}

jenkinsfile:

@Library("jenkinslib@master") _

withWrapNode("master") {

    stage("Build") {
        // do so
    }
}

Unkown interceptors for nested library calls when using explicit script invocation

I'm trying to invoke a script method explicitly via Script.invokeMethod() in order to avoid the usage of a declarative pipeline which would be a lot of overhead for my use case. The script contains multiple library steps which have been registered through an implicit shared library and PipelineTestHelper.registerSharedLibrary() . My issue is that nested library calls don't recognize the registered method interceptors from PipelineTestHelper.registerAllowedMethod with the exception:

groovy.lang.MissingMethodException: No signature of method: handleStepErrors.echo() is applicable for argument types: (org.codehaus.groovy.runtime.GStringImpl) values: [--- END LIBRARY STEP: publishTestResults.groovy ---]
Possible solutions: each(groovy.lang.Closure), wait(), run(), run(), find(), any()

Test using spock framework

how can i test jenkins pipline using spock framework
my spec look like this which extends Specification from spock framework

class MyFirstSpecification extends Specification {

}

since to use JenkinsPipelineUnit i need to extend BasePipelineTest.

Utils.groovy has @Grab depedency which breaks build inside firewall

The Utils class (src\test\resources\libs\commons@master\src\net\courtanet\jenkins\Utils.groovy) that is used in unit tests for the Shared Libraries uses a @grab dependency that makes it unnecessarily difficult to reproduce the build behind a firewall without changing the checked in code or adding cumbersome configuration options specific to the build. Since unit tests should be self contained either the script Grab dependency should use a @GrabResolver referencing a jar file that has been downloaded as part of the Gradle build, or the test case example should be rewritten to not need an external dependency (which in this case is not significant to the JenkinsPipelineUnit functionality under test.)

Write Wiki pages for how-to and troubleshooting

Proposed by @headcrabmeat in #50

Discussions in issues are ok but when the issue is closed it is not easily available to newcomers.

Write how-to pages resuming the solutions described in issues and troubleshooting for common issues and misunderstandings.

How are environment variables injected?

My Jenkinsfile uses environment variables like this:

username = env.USERNAME

I wonder how "env" can be mocked up in unit tests.
Currently, it gives an error:

Caused by: groovy.lang.MissingPropertyException: No such property: env for class: test

Mention JENKINS-33925

This repository seems to be addressing some or all of what has long been tracked as JENKINS-33925, so it would be a good idea to mention that somewhere in documentation.

Cannot register pipeline method names that are also Groovy keywords

I cannot find a way to register a pipeline method that is also a Groovy keyword.

For example:
registerAllowedMethod("sleep", [Integer.class], { println "SLEEP!!!"})

Then in my Jenkins pipeline under tests I have this to sleep 30 seconds:
sleep 30

The interceptors do not seem to pick it up, my registered closure is not called.

I found it as I was analysing my pipeline call-stacks while generating regressions. I think in the unit tests it is just doing a Groovy sleep(30) for 30 milliseconds so it is pretty harmless. Just wondered if there is a way to do this or if it is a limitation we have with this technique? Normally, anything is possible with Groovy so may be there is a way.

register allowed method which accept a Gstring

Following is the testcase using spock

def "test pipeline"() {
    setup:
    ....

    helper.registerAllowedMethod("node", [String.class, Closure.class], null)
    helper.registerAllowedMethod("node", [Closure.class], null)
    addEnvVar('workerLabel','testworker')
    when:
       loadScript('Jenkinsfile')
    then:
    println helper
}

my jenkinsfile having a node method with Gstring.

node("${env.workerLabel}") {
        if ((someProperties != null) && (anotherProperties != null)) {
            getEnvironmentsBuildNumbers(someProperties, anotherProperties)
        }
        logParser()
   }

while executing the testcase am getting MissingMethodException.

   groovy.lang.MissingMethodException: No signature of method: jenkinspipeline.node() is applicable for argument types: (org.codehaus.groovy.runtime.GStringImpl, jenkinspipeline$_call_closure2) values: [testworker, jenkinspipeline$_call_closure2@6d6bc158]
   Possible solutions: notify(), wait(), run(), call(), run(), any()

Bad null argument handling

If you do so:

def myMethod(String foo) {
  return 'bar'
}

myMethod null

Then you get : java.lang.IllegalArgumentException: wrong number of arguments.
We need a way to make the difference between a null value of an argument and its non-existence.

Provide a way to cache the PipelineTestHelper and dependent libraries

The PipelineTestHelper does a lot of heavy lifting e.g. loading a library from various sources. Since the library content doesn't change during the test execution it would be good to provide a way to cache this helper and it's dependent libraries to speed up the test execution and reducing the I/O effort.

Can I access the real "dir", "sh" etc. in my pipeline?

Hey,

I see that many pipeline steps are mocked by functions doing nothing (for example dir, sh etc.),

Now I want to test a groovy library which uses these steps, and I want it to really execute them.

How should I do that? Can I somehow access existing implementations of these commands that really do something? Do I have to implement them myself?

Thanks!
Nathan

Cannot call shared library steps with more than one parameter

Hey,
It seems that I cannot call steps from a shared library, if they have more parameters. Example:

vars/onePar.groovy

def call(par) {
}

vars/twoPar.groovy

def call(par1,par2) {
}

When I now try to test this I can call "onePar". But with twoPar I get:

No signature of method: com.lesfurets.jenkins.unit.PipelineTestHelper$_setGlobalVars_closure9$_closure18$_closure19.call() is applicable for argument types: (java.lang.String, java.lang.String) values: [par1, par2, ]
Possible solutions: any(), any(), grep(), find(), dump(), collect()
	at TestScripts.testscript(TestScripts.groovy:121)

Is the number of parameters somehow hardcoded? What could this be?

Thanks!

methodInterceptor closure passed to Script invokeMethod has wrong signature?

https://github.com/lesfurets/JenkinsPipelineUnit/blob/4c86c064146e45185a1621062fd0257bc293b4bf/src/main/groovy/com/lesfurets/jenkins/unit/PipelineTestHelper.groovy#L130

I updated to JenkinsPipelineUnit v1.1 and found that my mocks are being called with single element arrays where I might expect say just a string (e.g args value of ["foo"] instead of "foo").

The invokeMethod signature for Script is defined here:
http://docs.groovy-lang.org/2.4.11/html/gapi/groovy/lang/Script.html

The expected signature is:
public Object invokeMethod(String name, Object args)

But the signature for what is used in the helper is passing args as an array:
public methodInterceptor = { String name, Object[] args ->

It affected my project here where I get args using Spock mocks and have to de-reference the array:
https://github.com/macg33zr/pipelineUnit/blob/52eae8392f4ddb1460988ea1b31ecb0d301a7fbb/pipelineTests/groovy/tests/job/JenkinsfileTestSpec.groovy#L30

It is a bit subtle and can be worked around (change all tests though!).

What do you think is this an issue or a feature?

Best regards,
Bill

Question: Testing Shared Library definition using JenkinsPipelineUnit

I (finally) released the first version of https://github.com/mkobit/jenkins-pipeline-shared-libraries-gradle-plugin , which is a Gradle plugin intending to aid in the authoring of shared libraries. I'm sure an additional plugin could be added to support consumers of JenkinsPipelineUnit for build pipeline authors that also want to test their pipeline.

I wrote up an example library at https://github.com/mkobit/jenkins-pipeline-shared-library-example, but was struggling in writing a unit test against the local shared library functionality (at https://github.com/mkobit/jenkins-pipeline-shared-library-example/blob/master/test/unit/groovy/com/mkobit/libraryexample/JPUExampleSpec.groovy right now).

Any suggestions or example to look at for using this as a test harness for Shared Library definition?

Help needed - testing with an implicitly loaded global library

Hello,

I'm trying to add the simplest of tests to my existing (non-declarative) pipelines. As someone who is not a Java developer this is all rather daunting but I seem to, slowly, be getting somewhere! I have unfortunately ran into what seems like a road block. My company currently has a single shared library which is loaded implicitly by Jenkins. We use the library as follows:

def slack = new com.blah.jenkins.util.slack()

(...)

node {
  slack.notification(slackChannel, "<${env.BUILD_URL}|${env.BUILD_NUMBER}> failed :disappointed:")
}

I believe I've got my helper configured correctly (which is set to implicitly load the shared library from a remote git repo) however am running into a problem during the compile stage, not during the test stage. All the new calls to the shared library fail (as it's not been loaded - this is the pre-test compilation stage):

[ERROR] 	def slack = new com.blah.jenkins.util.slack()
[ERROR] 	                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ERROR] Groovy:unable to resolve class com.blah.jenkins.util.slack

I've tried adding an explicit @Library annotation but that resulted in a compile-time error as well:

[ERROR] 	@Library('common')
[ERROR] 	 ^^^^^^^
[ERROR] Groovy:unable to resolve class Library ,  unable to find class for annotation

I've had a look at a few example repos but these never seem to include any extra compile-time dependencies to make the @Library annotation work (ideally, if possible, I'd like to keep the existing code as-is ie. with implicit library loading and without annotations - is there a way to do this?).

Thanks in advance!

How to test DSL vars?

This library is great and really useful for testing Jenkins Pipelines. I'm using it to test our shared library for Jenkins and having some success. I'm setting up the tests with this:

  @Rule
  public TemporaryFolder scriptsFolder = new TemporaryFolder()

  @Before
  void setup() {
    setUp()
    FileUtils.copyDirectory(new File('.'), scriptsFolder.newFolder('sharedlib@master'))
    def library = library()
            .name('sharedlib')
            .retriever(localSource(scriptsFolder.root.absolutePath))
            .targetPath(scriptsFolder.root.absolutePath)
            .defaultVersion("master")
            .allowOverride(true)
            .implicit(false)
            .build()
    helper.registerSharedLibrary(library)
  }

When testing a simple var, this works fine:

// vars/gitCommit.groovy
import com.sharedlib.Scm

String call() {
  env.GIT_COMMIT ?: Scm.current().commit()
}
  def SHA = '96cd94a34dda6ce6a1ad51eeb4ea68769be45c00'

  @Test
  void testFindsInEnvironmentVariable() throws Exception {
    binding.setVariable('env', ['GIT_COMMIT' : SHA])
    loadScript("src/main/jenkins/pythonProject.jenkins")
    assertTrue(helper.callStack.findAll { call ->
        call.methodName == "sh"
    }.any { call ->
        callArgsToString(call).contains("echo $SHA")
    })
  }

However, if I try to bind a method to var that uses a DSL method, I can't seem to get it to locate the variable:

// vars/testPython.groovy
def call(body) {
  def config = [:]
  body.resolveStrategy = Closure.DELEGATE_FIRST
  body.delegate = config
  body()

  def sources = config.sources
  node {
    sh "nosetests --someflags ${sources}"
  }
}
  @Test
  void testPythonBlock() throws Exception {
    helper.registerAllowedMethod("node", [Map, Closure], null)
    loadScript("src/main/jenkins/pythonProject.jenkins")
  }
groovy.lang.MissingMethodException: No signature of method: testPython.node() is applicable for argument types: (testPython$_call_closure1) values: [testPython$_call_closure1@27adc16e]
Possible solutions: use([Ljava.lang.Object;), notify(), wait(), run(), run(), dump()
	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:58)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:81)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166)
	at testPython.call(testPython.groovy:22)

How can I bind these methods to vars?

Thanks.

SharedLib functions with more than one parameter not supported

Hello,
I currently try to test my shared library with JenkinsPipelineUnit. I always get the error message

groovy.lang.MissingMethodException: No signature of method: com.lesfurets.jenkins.unit.PipelineTestHelper$_setGlobalVars_closure9$_closure18$_closure19.call() is applicable for argument types: (java.lang.String, java.lang.String) values: [Some, String]

To avoid issues in my code I took from your commons library the sayHello.groovy and add it to my SharedLib. It is working if the call is like original:

//In Jenkinsfile:
sayHello('Some')

// in sayHello.groovy:
def call(String name = 'human') {

but if I change sayHello to have two parameters I get the mentioned error:

//In Jenkinsfile:
sayHello('Some', 'String')

// in sayHello.groovy:
def call(String name = 'human', String secondname = 'some') {

first lines of stacktrace:

MethodSignature{ name='sayHello', args=[class java.lang.String, class java.lang.String] }=com.lesfurets.jenkins.unit.PipelineTestHelper$_setGlobalVars_closure9$_closure18$_closure19@6caf0677
groovy.lang.MissingMethodException: No signature of method: com.lesfurets.jenkins.unit.PipelineTestHelper$_setGlobalVars_closure9$_closure18$_closure19.call() is applicable for argument types: (java.lang.String, java.lang.String) values: [Some, String]
Possible solutions: any(), any(), find(), collect(), grep(), dump()

java.lang.NullPointerException: Cannot get property 'Stage' on null object

	at org.codehaus.groovy.runtime.NullObject.getProperty(NullObject.java:60)
	at org.codehaus.groovy.runtime.InvokerHelper.getProperty(InvokerHelper.java:172)
	at org.codehaus.groovy.runtime.DefaultGroovyMethods.getAt(DefaultGroovyMethods.java:257)
	at org.codehaus.groovy.runtime.dgm$241.invoke(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoMetaMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:251)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.NullCallSite.call(NullCallSite.java:35)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)

Then I also changed my actual function of my shared lib to have only one parameter (hardcoded the others) and suddenly my unit test is running smooth.

Can you please check what could cause this? Thank you!

Cheers

Cannot invoke method if the script file name contains hyphens

For example, we have a scripts with the name pipeline-script.Jenkinsfile
And it contains the following code

def hello() {
    println "hello"
}
hello()

If we invoke loadScript('pipeline-script.Jenkinsfile') in tests we will get java.lang.ClassFormatError: Illegal class name "pipeline-script$hello" in class file pipeline-script$hello
However if we remove a hyphen in the script name(pipelinescript.Jenkinsfile) it will run without exceptions.

Jenkins pipeline plugin allows using scripts with any characters. So if our scripts contains hyphens we have to rename them for testing.

CPS tests fail for parallel with failFast: true

When using BasePipelineTestCPS with parallel and a task Map which includes failFast: true, I get this exception:

com.cloudbees.groovy.cps.impl.CpsCallableInvocation
	at sun.reflect.GeneratedConstructorAccessor59.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
	at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:263)
	at Jenkinsfile.run(Jenkinsfile)
	at TestJenkinsfile.should_execute_without_errors(TestJenkinsfile.java:75)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

I assume it's because of CPS, but nothing in there really explains the exception. However, if I use printCallStack() the call stack shows this:

Jenkinsfile.echo(ERROR integrationTests: groovy.lang.MissingMethodException: No signature of method: com.lesfurets.jenkins.unit.cps.PipelineTestHelperCPS$_closure1$_closure4.doCall() is applicable for argument types: (java.lang.String, java.lang.Boolean) values: [failFast, true] Possible solutions: doCall(java.lang.String, groovy.lang.Closure), findAll(), findAll())

This is the only documentation I can currently find for failFast: jenkinsci/pipeline-plugin#88

I think this may be legal syntax too: parallel tasksMap, failFast: true
but it fails with this call stack:

Jenkinsfile.echo(ERROR integrationTests: groovy.lang.MissingMethodException: No signature of method: Jenkinsfile.parallel() is applicable for argument types: (java.util.LinkedHashMap, java.util.LinkedHashMap) values: [[failFast:true], [JBOSSEAP:com.cloudbees.groovy.cps.impl.CpsClosure@3b08f438]])

Testing Shared Libraries that use Jenkins commands

I have been trying to test a shared library and went the route of wrapping it in a test pipeline and using this framework to exercise it. The library has a method that looks like

def getBuildProperties(){ return readFile('file.properties')} where readFile is a Jenkins command.

My test case is set up to register the method like so,

	helper.registerAllowedMethod(method("readFile", String.class), { file ->
	    return FileUtils.readFileToString(new File(file), Charset.forName("UTF-8"))
	})

The pipeline looks like


def execute() {
	def util = new Util()

	node() {
	
        stage('test') {
          readFile('file.properties') // works
          util.getBuildProperties() // does not work
        }
    }
}

The error I get is,

groovy.lang.MissingMethodException: No signature of method: com.company.Util.readFile() is applicable for argument types: (java.lang.String) values: [file.properties]
...

Is there something I'm missing here or is this not supported?
Is there a mechanism for testing a shared library in a standard unit test with the ability to have the Jenkins commands available, mocked or otherwise?

Include some dead simple example for a Maven project

Hello,

I'm trying to play with your tool, which seems very neat :-). But I immediately hit a wall, I'm using a pom.xml and created a simple JUnit, but for some reason I get an NPE on the gse variable from https://github.com/lesfurets/JenkinsPipelineUnit/blob/660d8dc51266e50f2aa741ce9fe173342737333e/src/main/groovy/com/lesfurets/jenkins/unit/PipelineTestHelper.groovy#L53

Just trying to simply load an existing script

public class AstroTest extends BasePipelineTest {
    @Test
    public void bim() throws Exception {
        loadScript("Jenkinsfile"); // fails with NPE
        System.out.println("WAT");
    }
}

But get the following stack trace:

	at com.lesfurets.jenkins.unit.PipelineTestHelper.loadScript(PipelineTestHelper.groovy:204)
	at com.lesfurets.jenkins.unit.PipelineTestHelper$loadScript$4.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
	at com.lesfurets.jenkins.unit.BasePipelineTest.loadScript(BasePipelineTest.groovy:83)
	at com.cloudbees.astro.AstroTest.bim(AstroTest.java:9)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

I guess it's something like it does not find the script, but I've no clue about where to put it or so. It would be great if the gse nullity would be checked to provide possibly more guidance (failing on https://github.com/lesfurets/JenkinsPipelineUnit/blob/660d8dc51266e50f2aa741ce9fe173342737333e/src/main/groovy/com/lesfurets/jenkins/unit/PipelineTestHelper.groovy#L204 because gse is null in this case)

I'm trying other combinations in the meantime.

Thanks!

Race condition with multiple Global Libraries?

I added a questionmark behind the topic, because I am not sure what happens.

We have a pipeline to create a build for technology X; since we are DRY we have a CORE-pipeline that contains shared bits of code that are used for all technologies.

This is in the @before of my Unittest class:

helper.registerSharedLibrary(library('pipeline-core')
                .retriever(GitSource.gitSource('https://something/core'))
                .targetPath(coreFolder.path)
                .defaultVersion("master")
                .allowOverride(true)
                .implicit(false)
                .build())

helper.registerSharedLibrary(library('pipeline-X')
                .retriever(GitSource.gitSource('https://something/X'))
                .targetPath(coreFolder.path)
                .defaultVersion("master")
                .allowOverride(true)
                .implicit(false)
                .build())

And then the test itself:

def script = loadScript("src/main/jenkins/pipeline.jenkins")

printCallStack()

pipeline.jenkins looks like this:

@Library(['pipeline-core','pipeline-X']) _
runPipelineFromX()

Now each time I run, one of these happens:

  • the @Library call fails, saying that it cannot load both pipeline-core and pipeline-X
  • It cannot find a number of dependencies of pipeline-X, all of these dependencies are part of pipeline-core
  • it starts running the pipeline and fails on one of the lines in there

The 3rd option is expected (I am still working on the mocking/binding)
The 1st option tells me sometimes the libraries are not done with loading
The 2nd option tells me that pipeline-X is loaded but pipeline-core was not finished yet

So my questions:

  • to me this looks like a race-condition, because the effect are random. Am I right? Is this related to multiple shared libraries?
  • If so: is there a way to tell the unittest to wait until both libraries are loaded? I am quite new in the world of Java, and I have read your code but I couldn't find a way to do this.

How to integrate JenkinsPipelineUnit within IntelliJ

Greeting,

I am trying to add JenkinsPipelineUnit into my project. I am having trouble with some setups.

Here is part of build.gradle:

sourceSets {
    main {
        groovy {
            srcDirs = ['vars', 'src']
        }
    }
    test {
        groovy {
            srcDirs = ['test/groovy']
        }
    }
}

As you can see, srcDirs for main are vars and src. I did this because I want IntelliJ to be aware of all the source code so it can do magic stuff(like jump into the source code) for me.

However, when I try to run tests, I get an error:

java.lang.ExceptionInInitializerError
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

This error will go away if I change main srcDirs to only 'src'. My guess is if I include 'vars' in source dir, somehow the test framework will not treat them as global libs instead of normal classes.

But if I remove that, IntelliJ cannot recognize all files in vars so the code reference functionality will stop working.

I am wondering have anybody seen this kind of problem before? maybe in another IDE?

Thanks

Help needed to test/mock a line

I am trying to unit test artifactory upload for the line below:

def artifactoryServer = script.Artifactory.server 'my_artifactory'

but I am unable to make this line work via helper.setVariable() or helper.registerAllowedMethod()
Any help on this is much appreciated?

Thanks

Could not initialize class with @Grab annotation in shared library

When I tried to run a test with our pipeline shared library, it failed to load any pipeline shared script with @ Grab annotation with error messages below. And once I removed the @ Grab annotation my test succeeded. Do you have any idea why this is happening?

General error during conversion: java.lang.NoClassDefFoundError: Could not initialize class inputWithTimeout

java.lang.RuntimeException: java.lang.NoClassDefFoundError: Could not initialize class inputWithTimeout
    at org.codehaus.groovy.control.CompilationUnit.convertUncaughtExceptionToCompilationError(CompilationUnit.java:1089)
    at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1067)
    at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:591)
    at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:569)
    at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:546)
    at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
    at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
    at groovy.util.GroovyScriptEngine$ScriptClassLoader.doParseClass(GroovyScriptEngine.java:256)
    at groovy.util.GroovyScriptEngine$ScriptClassLoader.parseClass(GroovyScriptEngine.java:243)
    at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
    at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:211)
    at groovy.util.GroovyScriptEngine.loadScriptByName(GroovyScriptEngine.java:568)
    at groovy.util.GroovyScriptEngine$loadScriptByName$0.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    at com.lesfurets.jenkins.unit.PipelineTestHelper.loadScript(PipelineTestHelper.groovy:323)
    at com.lesfurets.jenkins.unit.PipelineTestHelper$loadScript$1.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
    at com.lesfurets.jenkins.unit.BasePipelineTest.loadScript(BasePipelineTest.groovy:99)
    at com.lesfurets.jenkins.unit.BasePipelineTest$loadScript.callCurrent(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166)
    at com.indeed.pipeline.TestIvyUpdaterJenkinsfile.testIvyUpdaterMainline(TestIvyUpdaterJenkinsfile.groovy:46)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:114)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:57)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:66)
    at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
    at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
    at com.sun.proxy.$Proxy3.processTestClass(Unknown Source)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:109)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:147)
    at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:129)
    at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
    at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
    at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:46)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class inputWithTimeout
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
    at org.codehaus.groovy.reflection.CachedConstructor.doConstructorInvoke(CachedConstructor.java:77)
    at groovy.lang.MetaClassImpl.invokeConstructor(MetaClassImpl.java:1717)
    at groovy.lang.MetaClassImpl.invokeConstructor(MetaClassImpl.java:1534)
    at groovy.lang.ExpandoMetaClass.invokeConstructor(ExpandoMetaClass.java:681)
    at org.codehaus.groovy.runtime.InvokerHelper.invokeConstructorOf(InvokerHelper.java:954)
    at org.codehaus.groovy.runtime.DefaultGroovyMethods.newInstance(DefaultGroovyMethods.java:15628)
    at com.lesfurets.jenkins.unit.global.lib.LibraryLoader$_doLoadLibrary_closure4$_closure9.doCall(LibraryLoader.groovy:98)
    at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024)
    at groovy.lang.Closure.call(Closure.java:414)
    at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:54)
    at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:124)
    at com.sun.proxy.$Proxy15.accept(Unknown Source)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
    at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
    at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
    at java.util.Iterator.forEachRemaining(Iterator.java:116)
    at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
    at com.lesfurets.jenkins.unit.global.lib.LibraryLoader$_doLoadLibrary_closure4.doCall(LibraryLoader.groovy:90)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024)
    at groovy.lang.Closure.call(Closure.java:414)
    at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:54)
    at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:124)
    at com.sun.proxy.$Proxy15.accept(Unknown Source)
    at java.util.ArrayList.forEach(ArrayList.java:1249)
    at com.lesfurets.jenkins.unit.global.lib.LibraryLoader.doLoadLibrary(LibraryLoader.groovy:79)
    at com.lesfurets.jenkins.unit.global.lib.LibraryLoader.loadLibrary(LibraryLoader.groovy:62)
    at com.lesfurets.jenkins.unit.global.lib.LibraryAnnotationTransformer$_call_closure1.doCall(LibraryAnnotationTransformer.groovy:77)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024)
    at groovy.lang.Closure.call(Closure.java:414)
    at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:54)
    at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:124)
    at com.sun.proxy.$Proxy15.accept(Unknown Source)
    at java.util.HashMap$EntrySet.forEach(HashMap.java:1043)
    at com.lesfurets.jenkins.unit.global.lib.LibraryAnnotationTransformer.call(LibraryAnnotationTransformer.groovy:73)
    at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1053)
    ... 71 more

Failing to test a pipeline library

Hi guys,

Thanks for this framework which seems very nice but I'm failing to use it right now. So I have a pipeline library located in /vars/xwikiModule.groovy (content can be seen at https://github.com/xwiki/xwiki-jenkins-pipeline/blob/master/vars/xwikiModule.groovy) and I'm trying to test it using this unit test:

package org.xwiki.jenkins.test;

import java.io.File;

import org.junit.BeforeClass;
import org.junit.ClassRule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;

import com.lesfurets.jenkins.unit.cps.BasePipelineTestCPS;
import com.lesfurets.jenkins.unit.global.lib.LibraryConfiguration;

import static com.lesfurets.jenkins.unit.global.lib.LocalSource.*;
import static com.lesfurets.jenkins.unit.global.lib.LibraryConfiguration.*;

public class PipelineTest extends BasePipelineTestCPS
{
    @ClassRule
    public static TemporaryFolder tmpFolder = new TemporaryFolder();

    private static File temp;

    @BeforeClass
    public static void init() throws Exception
    {
        temp = tmpFolder.newFolder("xwiki");
    }

    @Test
    public void execute() throws Exception
    {
        super.setUp();

        LibraryConfiguration library = library()
            .name("XWiki")
            .retriever(localSource("/Users/vmassol/dev/xwiki/xwiki-jenkins-pipeline/vars/xwikiModule.groovy"))
            .targetPath(temp.getPath())
            .defaultVersion("master")
            .allowOverride(true)
            .implicit(true)
            .build();
        getHelper().registerSharedLibrary(library);

        loadScript("example.jenkins");
        printCallStack();
    }
}

The example.jenkins file contains:

@Library("XWiki@master")

xwikiModule {
}

It's failing with:

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
file:/Users/vmassol/dev/xwiki/xwiki-jenkins-pipeline/example.jenkins: 23: unexpected token: xwikiModule @ line 23, column 1.
   xwikiModule {
   ^

1 error


	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
	at org.codehaus.groovy.control.ErrorCollector.addFatalError(ErrorCollector.java:150)
	at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:120)
	at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:132)
	at org.codehaus.groovy.control.SourceUnit.addError(SourceUnit.java:360)
	at org.codehaus.groovy.antlr.AntlrParserPlugin.transformCSTIntoAST(AntlrParserPlugin.java:145)
	at org.codehaus.groovy.antlr.AntlrParserPlugin.parseCST(AntlrParserPlugin.java:111)
	at org.codehaus.groovy.control.SourceUnit.parse(SourceUnit.java:237)
	at org.codehaus.groovy.control.CompilationUnit$1.call(CompilationUnit.java:167)
	at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:931)
	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:593)
	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:569)
	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:546)
	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
	at groovy.util.GroovyScriptEngine$ScriptClassLoader.doParseClass(GroovyScriptEngine.java:256)
	at groovy.util.GroovyScriptEngine$ScriptClassLoader.parseClass(GroovyScriptEngine.java:243)
	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:212)
	at groovy.util.GroovyScriptEngine.loadScriptByName(GroovyScriptEngine.java:566)
	at groovy.util.GroovyScriptEngine$loadScriptByName$1.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
	at com.lesfurets.jenkins.unit.PipelineTestHelper.loadScript(PipelineTestHelper.groovy:323)

Anyone what I'm doing wrong? thanks!

Compiling pipeline DSL as Groovy class

Hello,

I have a generic Jenkins pipeline question, not entirely related to this repo.
I guess what I want to know is "if calling pipeline steps is possible outside the context of Jenkinsfile and Jenkins".
I would love to be able to compile shared pipeline libraries as normal Groovy code, but I get stuck all the time with dependencies and method signature issues.

I have the following code:

package com.mycode;

def read(fileName) {
  return libraryResource(fileName)
}

return this

which wraps the libraryResource step from the Jenkins pipeline DSL.
I wonder if there's a way to compile this class as a regular Groovy class? without being called by Jenkinsfile.

Thanks! โค๏ธ ๐Ÿ™ ๐Ÿ™‡โ€โ™€๏ธ

How to mock Jenkins commands (like steps.stash) to test shared library

Hi,
I am using JenkinsPipelineUtil to tests our library. I am not having success to mock Jenkins commands like steps.stash or steps.unstash

This is an other example of testing our library step

def call(Map parameters = [:]) {
    handleStepErrors (stepName: '_aTest', stepParameters: parameters) {
        def script = parameters.script
       def response = script.httpRequest httpMode: 'GET', authentication: parameters.credentials, url: parameters.url
        def delivery = script.readJSON text: response.content
        def result = delivery[0].DELIVERY_EXT_GUID
        echo result
        steps.stash result
    }
}

Here I am not able to mock script.httpRequest, script.readJSON and steps.stash
Could you please advise how to mock this?
Thank you very much in advance.

Not able to bind the scm in util files

Not able to bind the variables like scm inside main file calling other util files sample code
def call() {
String stage_pkg_prefix="com.wellsfargo.sharedlib.stages."
def mandatoryStages= ["Checkout","Build"]
def buildNumber, pipelineProperties, appProperties
def stageStartTime = new Date()
//def scmUtil = new ScmUtils()
def scmUtil = new ScmUtils()
//def utils = load "./loadLibraryResource.groovy"
// def scripta = load("./loadLibraryResource.groovy");
//pipelineProperties=utils.call("resources/common/pipelineProperties.groovy");
def lr=new loadLibraryResource()
def root="/pipelineProperties.groovy"
try{
node{
pipelineProperties = "abc"
stage("Checkout"){
node{
println "Executing Stage: Checkout"
def appURL
appURL = scmUtil.getBranchURL()
println "Executing Stage: appurl${appURL}"
println "Branch URL: ${appURL} ${pwd()}"
if(scmUtil.isGitScm()){
scmUtil.gitCheckOut(pipelineProperties.getRepoCredentialsID(), appURL,"develop");
println "Git Clone Completed"
}else{
scmUtil.subVersionCheckOut(pipelineProperties.getRepoCredentialsID(), appURL)
}
}
}
here scmUtil file having scm command which I am not able to bind it.

Cannot load a step when testing a shared library

I am testing my Jenkins shared library, in https://github.com/elifesciences/elife-jenkins-workflow-libs/blob/master/src/test/groovy/TestElifeNewRelicEnsureStatus.groovy I execute, in the test:

def script = loadScript("vars/elifeNewRelicEnsureStatus.groovy")
System.out.println(script);

I get this output:

TestExampleJob > should_execute_without_errors STANDARD_OUT
    null

What is the correct way to load a shared step and execute it? I saw some return this statements in other places but they return a complex object, doesn't seem to solve the problem.

Unable to test pipelines using Global Libraries

I have many pipelines that include a global library @Library('[email protected]')
We wrap some usage, such as Docker support. The pipelines use that in their code.
In the unit test, I am unable to overwrite the variables that get used in the library.

Pipeline Code:

def unitTestStep(data) {
  node {
    dockerSupport.insideContainer('docker.artifactory.company.com/company/alpine-docker-mvn-ci:0.1.0') {
    echo 'Running with Maven'
    sh "mvn test -P unit-tests -Dparam=${data}"```
  }
}

Unit Test Code:

void docker_example() {
    binding.setVariable("buildConfiguration", "Sample Build Configuration")
    def script = loadScript("path/to/pipelineCode.groovy")
    script.unitTestStep("This is my data. There are many like it, but this one is mine.")
    printCallStack()
}

Inside of dockerSupport.insideContainer there is code being called that uses buildConfiguration that is not getting set. The line binding.setVariable("buildConfiguration", "Sample Build Configuration") does not override this.

I'm unsure how to mock these things out to move forward.

Unable to access default/new methods in helper and binding variables is null

Hi There,

I am trying to write a unit test class for testing shared library code:
`
class OpsTest extends BasePipelineTest {

@Override
@Before
void setUp() throws Exception {

    super.setUp()
    helper.registerAllowedMethod("send", [Map.class], null)
 
    // Defines the previous execution status      
    binding.getVariable('currentBuild').currentResult = [result: 'UNSTABLE']
    binding.setVariable('env',[JOB_NAME: 'test'])
}

@Test
void testSendNotification() {
    def script = helper.loadScript("opsTest.jenkins",binding)
   script.execute()
    printCallStack()
    //assertToString(stud.Display(), expected)
}

}
`

However, I am getting error:
groovy.lang.MissingMethodException: No signature of method: ops.send() is applicable for argument types: (LinkedHashMap) values: [.....]

I am getting the same exception for any default methods like 'echo', 'junit'

Is it possible to ignore/mock methods e.g "send" that I do not wish to test as part of unit tests?
Is it possible to test the actual methods in the shared library without having to go via Jenkinsfile?

Thanks,
Richa

Code coverage of pipeline library - MissingMethodException

I'm attempting to analyze the unit test code coverage of our pipeline groovy scripts using jacoco. Right now only the groovy code under the src directory is getting analyzed and I believe the scripts under vars aren't getting analyzed because vars isn't part of the source class path.

When adding the vars directory to the sourceSets in build.gradle (see below) some of my tests fail with an error like groovy.lang.MissingMethodException: No signature of method: static hello.world() is applicable for argument types: () values: []. This error seems to only occur when my pipeline scripts make explicit static method calls (e.g. hello.world()). Is this a bug or user error? Any possible workarounds? Thanks in advance!

// build.gradle

sourceSets {
  main {
    groovy {
      srcDirs = ['src', 'vars']
      exclude '**/test/**'
    }
  }
}
// vars/hello.groovy

def world() {
  echo "hello, world!"
}

Testing Library Method that Accesses Build Variable

I have a Pipeline Library that accesses the build number of the current build. Something along the lines of the following method:

def getCurrentBuildNumber() { def output = env.getEnvironment().containsKey("BUILD_NUMBER") ? new String("$env.BUILD_NUMBER") : new String("") return output }

This works fine in a Jenkins Jobs, however I cannot figure out how to test this kind of thing. The problem seems to be related to env.

A simple test case setting the variable with

helper.setGlobalVars(new Binding(["BUILD_NUMBER":123]))

for the method above yields a groovy.lang.MissingPropertyException No such property: env when I call the above method. I assume that in an actual Jenkins instance this functionality is provided by an injected EnvActionImpl class, but I understand too little of Jenkins internals to actually know whats going on.

Is this a bug/missing feature or am I doing it wrong?

Generate workspace does not create the output directory

$ bazel run //generate_workspace -- -a com.lesfurets:jenkins-pipeline-unit:1.0 -o /tmp/test
INFO: Analysed target //generate_workspace:generate_workspace.
INFO: Found 1 target...
Target //generate_workspace:generate_workspace up-to-date:
  bazel-bin/generate_workspace/generate_workspace.jar
  bazel-bin/generate_workspace/generate_workspace
INFO: Elapsed time: 0.277s, Critical Path: 0.02s
INFO: Build completed successfully, 1 total action

INFO: Running command line: bazel-bin/generate_workspace/generate_workspace -a com.lesfurets:jenkins-pipeline-unit:1.0 -o /tmp/test
Jun 28, 2017 6:34:20 PM com.google.devtools.build.workspace.maven.Resolver traverseDeps
INFO: 	Downloading pom for com.lesfurets:jenkins-pipeline-unit:1.0
Jun 28, 2017 6:34:21 PM com.google.devtools.build.workspace.maven.Resolver traverseDeps
INFO: 	Downloading pom for org.codehaus.groovy:groovy-all:2.4.6
Jun 28, 2017 6:34:23 PM com.google.devtools.build.workspace.maven.Resolver traverseDeps
INFO: 	Downloading pom for com.cloudbees:groovy-cps:1.12
Jun 28, 2017 6:34:25 PM com.google.devtools.build.workspace.maven.Resolver traverseDeps
INFO: 	Downloading pom for com.google.guava:guava:11.0.1
Jun 28, 2017 6:34:26 PM com.google.devtools.build.workspace.maven.Resolver traverseDeps
INFO: 	Downloading pom for com.google.code.findbugs:jsr305:1.3.9
Jun 28, 2017 6:34:29 PM com.google.devtools.build.workspace.maven.Resolver traverseDeps
INFO: 	Downloading pom for commons-io:commons-io:2.5
Jun 28, 2017 6:34:33 PM com.google.devtools.build.workspace.maven.Resolver traverseDeps
INFO: 	Downloading pom for org.apache.ivy:ivy:2.4.0
Jun 28, 2017 6:34:33 PM com.google.devtools.build.workspace.output.BzlWriter write
SEVERE: Could not write /tmp/test/generate_workspace.bzl: /tmp/test/generate_workspace.bzl (No such file or directory)

Issue in mocking a script in shared library folder

I have the following script files aScript.groovy and bScript.groovy both are in shared library folder, these scripts will be called from a jenkinsFile in the project folder; it executed. well I need to mock bScript.groovy.
how can i mock this, i tried using helper.registerAllowedMethod("bScript", [], null) but its not mocking bScript, I also tried using metaclass bScript.metaClass.static.call = {return 'bar'} but its giving me groovy.lang.MissingPropertyException: No such property: bScript for class: aScriptSpec

aScript.groovy

 def call(){
    return bScript()
}

bScript.groovy

  def call(){
     return "bar"
 }

Transform the build gradle package into a plugin

I saw you'd added own gradle package.

I like the feature to test and debug Grape in test cases. I also would like to use it my tests.

Maybe it could be some how transformed into a gradle plugin. So I could not to copy part of your gradle configuration but to apply a plugin?

If you took it from any existed gradle plugin. Could you please share the name?

Support Declarative Pipeline

Filing that around for feedback/ideas, even if I'm also looking into this a bit, though really not an experienced Groovy dev. Only using it casually.

I think I should register the pipeline top function to org.jenkinsci.plugins.pipeline.modeldefinition.model.Root from pipeline-model-definition, but not sure what it would/could lead to yet.

Demo project

Hi, this is freaking awesome!!! Do you have a demo project?

MissingMethodException: No signature of method

In our jenkins shared library project we commonly reuse or invoke functions from other functions in the same library. I'm attempting to write unit tests for this scenario and keep hitting the MissingMethodException exception.

Here's a simplified example:

// file: vars/sayHello.groovy
def call(body) {
  def config = [:]
  body.resolveStrategy = Closure.DELEGATE_FIRST
  body.delegate = config
  body()

  echo 'say Hello!'
}

// file: vars/greetings.groovy
def call(body) {
  def config = [:]
  body.resolveStrategy = Closure.DELEGATE_FIRST
  body.delegate = config
  body()

  sayHello {}
}

// file: Jenkinsfile
@Library('sharedlib') _

node {
  greetings {}
}

When executing a test it throws:

groovy.lang.MissingMethodException: No signature of method: greetings.sayHello() is applicable for argument types: (greetings$_call_closure1) values: [greetings$_call_closure1@3407e97e]

Is this a limitation of the framework or am I doing something wrong?

RegressionTestHelper / TestNonRegression line endings issue developing on Windows

When developing on windows, when the call stack regression test file is generated it has Windows line endings (\r\n). After commit the regression test fails as the comparison does not normalise the line endings. The helper needs to normalise line endings.

In class RegressionTestHelper the method TestNonRegression it needs to do this:

assertThat(callStack.normalize()).isEqualTo(referenceFile.text.normalize())

How to test the when step in a stage

I want to test that the stage with my shared lib executes based on the when environment expression being true.

For ex:

pipeline {
    agent none
    stages {
        stage("Shared Lib 1") {
            agent any
            when {
                environment name:'SHARED_LIB_1', value: 'true'
            }
            steps {
                sharedLib1()
            }
        }
    }
}

This is what I was using:

helper.registerAllowedMethod('environment', [Map.class], {it -> (it.value == binding.getProperty('env')["SHARED_LIB_1"]) })
helper.registerAllowedMethod('when', [Closure.class], {c -> c.call()})

It always executes the stage for any value of SHARED_LIB_1 set in the environment.

How to go about testing the when condition in stage with environment expression? Thanks.

Cannot printCallStack for a parallel step with a complex map

I've suggested a test with a complex map like.

s = [s1:[name:"action1-name", body:{
      node() {
        sh 'sleep 3'
        error 'message'
      }
    }], s2:[name: "action2-name", body:{
        node() {
            sh 'sleep 4'
        }
    }]]

    parallel(
      "$s.s1.name":s.s1.body,
      "$s.s2.name":s.s2.body,
      failFast:true
    )

It seems that the script itself works fine, however I get the exception on printCallStack()
``
java.lang.ClassCastException: org.codehaus.groovy.runtime.GStringImpl cannot be cast to java.lang.String

How to mock a method with a specific parameter?

I mocked out fileExists method like this
registerJenkinsMethod('fileExists', [String.class], {c -> exists})
but this mocks fileExists for ALL of the files and there is not a way to tell for which file fileExists executed on.

Is there a way to mock method in a parameter-sensitive way?

Thanks!!! ๐Ÿ™

Way to override method in shared library?

I'm unit testing a shared library, which I load in the setUp into a variable library.

each (spock) test of the shared library then takes the form:

given: "a mock bar"
    def bar = Mock (Closure)
    helper.registerAllowedMethod ('bar', [String.class], bar)
when: "foo is called"
    library.foo()
then: "bar is called"
    1 * bar (_)

This works fine if bar is some jenkins method (e.g sh or something), but if bar is a different method in the same shared library then registration doesn't replace that bar with the mock for the test.

This means that I can only test the library itself as a unit (foo + bar + mock of bar's externals) rather than each method as a self contained unit (foo + mock of foo's externals including bar).

If bar is complex, then the mocking to exercise foo becomes correspondingly complex, when really all I want to confirm in the is that foo calls bar.

Is there a way to achieve the replacement of an arbitrary library method in this way? I know I can probably "load" the foo.groovy directly, but that requires replacing all of the potentially complex library dependencies that foo has rather than just the interesting bits.

How to register methods without arguments

I've been stuck on trying to register a method that has no arguments.
I get nullPointerException trying to do it the way below:
getHelper().registerAllowedMethod(method(methodName, null), { c -> "${methodName} executed successfully" })
How do I use registerAllowedMethod to mock methods without arguments?
๐Ÿ™ ๐Ÿ™ ๐Ÿ™ Any help is greatly appreciated ๐Ÿ™ ๐Ÿ™ ๐Ÿ™

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.