Skip to content

Allow overriding mvn commands in integration test #420

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 24, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 17 additions & 10 deletions ReadMe.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,13 @@ NOTE: if `cmd` is set, `goal`, `profile`, `skipTests` will have no effect.
itestPattern = '*KT'
}
```
- pass `cmd` parameter to override the `mvn` command to execute in integration test
```groovy
mavenIntegrationTest {
cmd = 'mvn -P openshift-it org.apache.maven.plugins:maven-failsafe-plugin:verify'
}
```
Note: All other flags are ignored if `mavenIntegrationTest` has `integrationTestCmd` parameter.
#### Merge and Wait for Pull Request

- adds a [merge] comment to a github pull request
Expand Down Expand Up @@ -248,7 +255,7 @@ NOTE: if `cmd` is set, `goal`, `profile`, `skipTests` will have no effect.
}
```
#### Update Maven Property Version
During a release involving multiple java projects we often need to update downstream maven poms with new versions of a dependency. In a release pipeline we want to automate this, set up a pull request and let CI run to make sure there's no conflicts.
During a release involving multiple java projects we often need to update downstream maven poms with new versions of a dependency. In a release pipeline we want to automate this, set up a pull request and let CI run to make sure there's no conflicts.

- performs a search and replace in the maven pom
- finds the latest version available in maven central (repo is configurable)
Expand All @@ -274,7 +281,7 @@ Automating this has saved us a lot of time during the release pipeline
}
```
#### Wait Until Artifact Synced With Maven Central
When working with open source java projects we need to stage artifacts with OSS Sonatype in order to promote them into maven central. This can take 10-30 mins depending on the size of the artifacts being synced.
When working with open source java projects we need to stage artifacts with OSS Sonatype in order to promote them into maven central. This can take 10-30 mins depending on the size of the artifacts being synced.

A useful thing is to be notified in chat when artifacts are available in maven central as blocking the pipeine until we're sure the promote has worked.

Expand Down Expand Up @@ -324,7 +331,7 @@ When a project is staged an array is returned and passed around functions furthe
}
```

One other important note is on the fabric8 project we don't use the maven release plugin or update to next SNAPSHOT versions as it causes unwanted noise and commits to our many github repos. Instead we use a fixed development `x.x-SNAPSHOT` version so we can easily work in development on multiple projects that have maven dependencies with each other.
One other important note is on the fabric8 project we don't use the maven release plugin or update to next SNAPSHOT versions as it causes unwanted noise and commits to our many github repos. Instead we use a fixed development `x.x-SNAPSHOT` version so we can easily work in development on multiple projects that have maven dependencies with each other.

Now that we don't store the next release version in the poms we need to figure it out during the release. Rather than store the version number in the repo which involves a commit and not too CD friendly (i.e. would trigger another release just for the version update) we use the `git tag`. From this we can get the previous release version, increment it and push it back without triggering another release. This seems a bit strange but it has been holding up and has significantly reduced unwanted SCM commits related to maven releases.

Expand Down Expand Up @@ -389,8 +396,8 @@ Now that we don't store the next release version in the poms we need to figure i
```
#### Git Tag

- tags the current git repo with the provided version
- pushes the tag to the remote repository
- tags the current git repo with the provided version
- pushes the tag to the remote repository

```groovy
gitTag {
Expand All @@ -415,9 +422,9 @@ __NOTE__ in order for images to be found by the remote OpenShift instance it mus

#### Deploy Remote Kubernetes

Deploys the staged fabric8 release to a remote Kubernetes cluster
Deploys the staged fabric8 release to a remote Kubernetes cluster

__NOTE__ in order for images to be found by the remote OpenShift instance it must be able to pull images from the staging docker registry. Noting private networks and insecure-registry flags.
__NOTE__ in order for images to be found by the remote OpenShift instance it must be able to pull images from the staging docker registry. Noting private networks and insecure-registry flags.

```groovy
node {
Expand Down Expand Up @@ -587,7 +594,7 @@ For this case you can combine add the docker template and the maven template tog
node('maven-and-docker') {
container(name: 'maven') {
sh 'mvn clean package fabric8:build fabric8:push'
}
}
}
}
}
Expand All @@ -598,7 +605,7 @@ The above is equivalent to:
mavenNode(label: 'maven-and-docker') {
container(name: 'maven') {
sh 'mvn clean package fabric8:build fabric8:push'
}
}
}
}

Expand All @@ -613,7 +620,7 @@ In the example above we can add release capabilities too, by adding the releaseT
mvn release:clean release:prepare
mvn clean release:perform
"""
}
}
}
}
}
Expand Down
5 changes: 5 additions & 0 deletions src/io/fabric8/Utils.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ String environmentNamespace(String environment) {
return ns + "-${environment.toLowerCase()}"
}

@NonCPS
String testNamespace() {
return getUsersNamespace();
}

/**
* Loads the environments in the default user namespace
*/
Expand Down
5 changes: 3 additions & 2 deletions vars/mavenCI.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,14 @@ def call(body) {
def goal = config.goal ?: "install"
def profile = config.profile ?: "openshift"
def skipTests = config.skipTests ?: false
def cmd = config.cmd ?: "mvn clean -B -e -U ${goal} -Dmaven.test.skip=${skipTests} -P ${profile}"
def buildCmd = config.buildCmd ?: "mvn clean -B -e -U ${goal} -Dmaven.test.skip=${skipTests} -P ${profile}"

def version = 'PR-' + getNewVersion {} + "-${env.BUILD_NUMBER}"

stage('Build + Unit test') {
// set a unique temp version so we can download artifacts from nexus and run acceptance tests
sh "mvn -U versions:set -DnewVersion=${version}"
sh cmd
sh buildCmd
}

def s2iMode = utils.supportsOpenShiftS2I()
Expand All @@ -63,6 +63,7 @@ def call(body) {

stage('Integration Testing') {
mavenIntegrationTest {
cmd = config.integrationTestCmd
environment = 'Test'
failIfNoTests = false
itestPattern = '*IT'
Expand Down
32 changes: 21 additions & 11 deletions vars/mavenIntegrationTest.groovy
Original file line number Diff line number Diff line change
@@ -1,36 +1,46 @@
#!/usr/bin/groovy

import io.fabric8.Utils

def call(body) {
Copy link
Contributor

@sthaha sthaha Aug 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about this

def mvnTestCommandForConfig(config) {
    def utils = new Utils()

    def envName = config.environment
    def kubeNS = "-Dfabric8.environment=${envName}"

    if (envName) {
        try {
            def ns = utils.environmentNamespace(envName)
            if (ns) {
                kubeNS = "-Dnamespace.use.existing=${ns}"
                echo "Running the integration tests in namespace : ${ns}"
            }
        } catch (e) {
            echo "ERROR: failed to find the environment namespace for ${envName} due to ${e}"
            e.printStackTrace()
        }
    }

    return "mvn \
      org.apache.maven.plugins:maven-failsafe-plugin:integration-test \
      -P openshift-it ${kubeNS} \
      -Dit.test=${config.itestPattern} \
      -DfailIfNoTests=${config.failIfNoTests}  \
      org.apache.maven.plugins:maven-failsafe-plugin:verify"
}

def call(body) {
    def utils = new Utils()

    if (utils.isDisabledITests()) {
        echo "WARNING: Integration tests DISABLED for these pipelines!"
        return
    }

    // evaluate the body block, and collect configuration into the object
    def config = [:]
    body.resolveStrategy = Closure.DELEGATE_FIRST
    body.delegate = config
    body()


    // execute integration test command if it is explicitly set and exit
    def cmd = "" config.cmd ?: mvnTestCommandForConfig(config)
    sh cmd
    junitResults(body);
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't tested this but the idea is quite simple ... The call either makes the cmd to run or uses the cmd passed to it and it is quite clear IMHO

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks more readable +1


if (new Utils().isDisabledITests()) {
echo "WARNING: Integration tests DISABLED for these pipelines!"
return
}

// evaluate the body block, and collect configuration into the object
def config = [:]
body.resolveStrategy = Closure.DELEGATE_FIRST
body.delegate = config
body()

def cmd = config.cmd ?: defaultTestCommand(config)
sh cmd
junitResults(body);
}

def defaultTestCommand(config) {
def utils = new Utils()

def envName = config.environment
def kubeNS = "-Dfabric8.environment=${envName}"

if (envName) {
// lets try find the actual kubernetes namespace
try {
def ns = utils.environmentNamespace(envName)
if (ns) {
kubeNS = "-Dnamespace.use.existing=${ns}"
echo "Running the integration tests in the namespace: ${ns}"
echo "Running the integration tests in namespace : ${ns}"
}
} catch (e) {
echo "ERROR: failed to find the environment namespace for ${envName} due to ${e}"
e.printStackTrace()
}
}

if (utils.isDisabledITests()) {
echo "WARNING: Integration tests are current DISABLED for these pipelines!"

} else {
sh "mvn org.apache.maven.plugins:maven-failsafe-plugin:integration-test ${kubeNS} -P openshift-it -Dit.test=${config.itestPattern} -DfailIfNoTests=${config.failIfNoTests} org.apache.maven.plugins:maven-failsafe-plugin:verify"

junitResults(body);
}
return "mvn \
org.apache.maven.plugins:maven-failsafe-plugin:integration-test \
org.apache.maven.plugins:maven-failsafe-plugin:verify \
-P openshift-it ${kubeNS} \
-Dit.test=${config.itestPattern} -DfailIfNoTests=${config.failIfNoTests}"
}