Skip to content

Allow overriding mvn commands in integration test #420

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 24, 2018

Conversation

hrishin
Copy link
Member

@hrishin hrishin commented Aug 10, 2018

This patch allows the mavenCI{} to over maven command in order
to allow mavenIntegrationTes{} to pass specific command to execute.

Related to: #412

@hrishin hrishin requested a review from bartoszmajsak August 10, 2018 07:45
@hrishin hrishin changed the title Allow overriding mvn commands Allow overriding mvn commands integration test Aug 10, 2018
@hrishin hrishin changed the title Allow overriding mvn commands integration test Allow overriding mvn commands in integration test Aug 10, 2018
@hrishin
Copy link
Member Author

hrishin commented Aug 10, 2018

@bartoszmajsak this would work for you?

@bartoszmajsak
Copy link
Contributor

bartoszmajsak commented Aug 10, 2018

@hrishin In theory it might, but in reality not quite.

In booster pipelines (but I also think in general use case) we call mavenCI{} to which you can pass cmd but it is a different cmd that the one you expect here. We would need to have a way to cascade custom integration tests command to this bit

mavenIntegrationTest {
environment = 'Test'
failIfNoTests = false
itestPattern = '*IT'
}

@hrishin
Copy link
Member Author

hrishin commented Aug 10, 2018

Then how we can provide the interface from Jenkins file?
By accepting the one more cmd command or flags these flags as a key-value pair or accept the mavenIntegrationTest as a closure or something else?

@bartoszmajsak

@bartoszmajsak
Copy link
Contributor

bartoszmajsak commented Aug 10, 2018

I think we need to be more specific with the arguments. I could think about following

mavenCI {
  deploy = 'mvn clean deploy -P openshift'
  integrationTests = 'mvn clean install-Dnamespace.use.current=false -DenableImageStreamDetection=true -P openshift-it'
}

then in the mavenCI.groovy you would have:

mavenIntegrationTest { 
     environment = 'Test' 
     integrationTests = config.integrationTests
 } 

then the follow up question is what do we do with other "hardcoded bits in there, namely

     failIfNoTests = false 
     itestPattern = '*IT' 

and alikes.

@@ -11,6 +11,8 @@ def call(body) {
def utils = new Utils()
def envName = config.environment
def kubeNS = "-Dfabric8.environment=${envName}"
def cmd = cmd ?: "mvn org.apache.maven.plugins:maven-failsafe-plugin:integration-test ${kubeNS} -P openshift-it -Dit.test=${config.itestPattern} -DfailIfNoTests=${config.failIfNoTests} org.apache.maven.plugins:maven-failsafe-plugin:verify"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be config.cmd ?: ...

@hrishin
Copy link
Member Author

hrishin commented Aug 13, 2018

@bartoszmajsak apart from integrationTestCmd, do we need to provide namespace argument as well?

@hrishin
Copy link
Member Author

hrishin commented Aug 13, 2018

With this patch now mavenCI would able to maven command to execute the IT.

mavenCI {
    integrationTestCmd = 'mvn -P openshift-it org.apache.maven.plugins:maven-failsafe-plugin:verify'
} 

@bartoszmajsak

@hrishin
Copy link
Member Author

hrishin commented Aug 13, 2018

I have tested this patch.
The Jenkinsfile for project https://github.com/hrishin/pr-test-vertex/pull/3/files

Jenkins logs for integration test. Appears like due to permission issue this test case got failed in order to create new project.

Running the integration tests in the namespace: hshinde
[Pipeline] echo
Loaded PipelineConfiguration PipelineConfiguration{jobNameToKindMap={}, ciBranchPatterns=[PR-.*], cdBranchPatterns=[master], cdGitHostAndOrganisationToBranchPatterns={}, disableITestsCD=false, disableITestsCI=false, useDockerSocketFlag=false}
[Pipeline] echo
Using branch PR-3
[Pipeline] sh
[hrishin_pr-test-vertex_PR-3-33KIIYNG4ZNY5GC3GT2TDHVG46HGZ3YD7G5L535DLP7LDIBCF5TQ] Running shell script
+ mvn clean install -Dnamespace.use.current=false -DenableImageStreamDetection=true -P openshift-it
Picked up _JAVA_OPTIONS: -Duser.home=/root/ -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -Dsun.zip.disableMemoryMapping=true -XX:+UseParallelGC -XX:MinHeapFreeRatio=5 -XX:MaxHeapFreeRatio=10 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -Xms10m -Xmx192m
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Vert.x - HTTP PR-v3c38d0d-2
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ pr-test-vertex ---
[INFO] Deleting /home/jenkins/workspace/hrishin_pr-test-vertex_PR-3-33KIIYNG4ZNY5GC3GT2TDHVG46HGZ3YD7G5L535DLP7LDIBCF5TQ/target
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-java-version) @ pr-test-vertex ---
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven-version) @ pr-test-vertex ---
[INFO] 
[INFO] --- buildnumber-maven-plugin:1.4:create (get-scm-revision) @ pr-test-vertex ---
[INFO] Executing: /bin/sh -c cd '/home/jenkins/workspace/hrishin_pr-test-vertex_PR-3-33KIIYNG4ZNY5GC3GT2TDHVG46HGZ3YD7G5L535DLP7LDIBCF5TQ' && 'git' 'rev-parse' '--verify' 'HEAD'
[INFO] Working directory: /home/jenkins/workspace/hrishin_pr-test-vertex_PR-3-33KIIYNG4ZNY5GC3GT2TDHVG46HGZ3YD7G5L535DLP7LDIBCF5TQ
[INFO] Storing buildNumber: 3c38d0d47226e7ad714cef8ec828a9fa308d7cad at timestamp: 1534175170708
[WARNING] Cannot get the branch information from the git repository: 
Detecting the current branch failed: fatal: ref HEAD is not a symbolic ref

[INFO] Executing: /bin/sh -c cd '/home/jenkins/workspace/hrishin_pr-test-vertex_PR-3-33KIIYNG4ZNY5GC3GT2TDHVG46HGZ3YD7G5L535DLP7LDIBCF5TQ' && 'git' 'rev-parse' '--verify' 'HEAD'
[INFO] Working directory: /home/jenkins/workspace/hrishin_pr-test-vertex_PR-3-33KIIYNG4ZNY5GC3GT2TDHVG46HGZ3YD7G5L535DLP7LDIBCF5TQ
[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] --- vertx-maven-plugin:1.0.15:initialize (vmp) @ pr-test-vertex ---
[INFO] 
[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ pr-test-vertex ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ pr-test-vertex ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/jenkins/workspace/hrishin_pr-test-vertex_PR-3-33KIIYNG4ZNY5GC3GT2TDHVG46HGZ3YD7G5L535DLP7LDIBCF5TQ/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:3.0.2:testResources (default-testResources) @ pr-test-vertex ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ pr-test-vertex ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /home/jenkins/workspace/hrishin_pr-test-vertex_PR-3-33KIIYNG4ZNY5GC3GT2TDHVG46HGZ3YD7G5L535DLP7LDIBCF5TQ/target/test-classes
[INFO] 
[INFO] --- maven-surefire-plugin:2.20:test (default-test) @ pr-test-vertex ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running io.openshift.booster.HttpApplicationTest
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Server started on port 8081
Server started on port 8081
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.902 s - in io.openshift.booster.HttpApplicationTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ pr-test-vertex ---
[INFO] Building jar: /home/jenkins/workspace/hrishin_pr-test-vertex_PR-3-33KIIYNG4ZNY5GC3GT2TDHVG46HGZ3YD7G5L535DLP7LDIBCF5TQ/target/pr-test-vertex-PR-v3c38d0d-2.jar
[INFO] 
[INFO] --- vertx-maven-plugin:1.0.15:package (vmp) @ pr-test-vertex ---
[INFO] Executing: /bin/sh -c cd '/home/jenkins/workspace/hrishin_pr-test-vertex_PR-3-33KIIYNG4ZNY5GC3GT2TDHVG46HGZ3YD7G5L535DLP7LDIBCF5TQ' && 'git' 'whatchanged' '--until=2018-08-13 15:46:31 +0000' '--date=iso' '--' '/home/jenkins/workspace/hrishin_pr-test-vertex_PR-3-33KIIYNG4ZNY5GC3GT2TDHVG46HGZ3YD7G5L535DLP7LDIBCF5TQ'
[INFO] Working directory: /home/jenkins/workspace/hrishin_pr-test-vertex_PR-3-33KIIYNG4ZNY5GC3GT2TDHVG46HGZ3YD7G5L535DLP7LDIBCF5TQ
[INFO] 
[INFO] --- maven-failsafe-plugin:2.20:integration-test (default) @ pr-test-vertex ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running io.openshift.booster.OpenShiftIT
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
CubeKubernetesConfiguration: 
  namespace = itest-fa83b848
  master.url = https://kubernetes.default.svc/
  namespace.lazy.enabled = true
  namespace.cleanup.enabled = true
  namespace.cleanup.timeout = 0
  namespace.cleanup.confirm.enabled = false
  namespace.destroy.enabled = true
  namespace.destroy.confirm.enabled = false
  namespace.destroy.timeout = 0
  wait.enabled = true
  wait.timeout = 480000
  wait.poll.interval = 5000
  ansi.logger.enabled = true
  env.init.enabled = true
  logs.copy = false
  cube.api.version = v1
  cube.trust.certs = true
  cube.fmp.build = false
  cube.fmp.build.disable.for.mvn = false
  cube.fmp.pom.path = pom.xml
  cube.fmp.debug.output = false
  cube.fmp.logs = true
  cube.fmp.build.options = 

CubeOpenShiftConfiguration: 
  keepAliveGitServer = false
  autoStartContainers = []
  proxiedContainerPorts = []
  portForwardBindAddress = 127.0.0.1
  openshiftRouterHttpPort = 80
  openshiftRouterHttpsPort = 443
  enableImageStreamDetection = true
  routerSniPort = 443
  templateProcess = true
  stratupTimeout = 600
  httpClientTimeout = 120

�[32mInitializing Session:fa83b848�[m
�[32mUsing Kubernetes at: https://kubernetes.default.svc/�[m
�[32mCreating project: itest-fa83b848�[m
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.842 s <<< FAILURE! - in io.openshift.booster.OpenShiftIT
[ERROR] io.openshift.booster.OpenShiftIT  Time elapsed: 2.841 s  <<< ERROR!
io.fabric8.kubernetes.clnt.v3_1.KubernetesClientException: Failure executing: POST at: https://kubernetes.default.svc/apis/project.openshift.io/v1/projectrequests. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. projectrequests.project.openshift.io is forbidden: User "system:serviceaccount:hshinde-jenkins:jenkins" cannot create projectrequests.project.openshift.io at the cluster scope: You may not request a new project via this API..

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR]   OpenShiftIT.io.openshift.booster.OpenShiftIT ? KubernetesClient Failure execut...
[INFO] 
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- maven-failsafe-plugin:2.20:verify (default) @ pr-test-vertex ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 41.218 s
[INFO] Finished at: 2018-08-13T15:46:41+00:00
[INFO] Final Memory: 29M/52M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.20:verify (default) on project pr-test-vertex: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/jenkins/workspace/hrishin_pr-test-vertex_PR-3-33KIIYNG4ZNY5GC3GT2TDHVG46HGZ3YD7G5L535DLP7LDIBCF5TQ/target/failsafe-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

@bartoszmajsak
Copy link
Contributor

@hrishin your parameters are not enough. Test wants to create namespace namespace = itest-fa83b848 and that's why it is failing. The whole idea here is to re-use user namespace, as the log at the very beginning suggests:

Running the integration tests in the namespace: hshinde

You can see the full command in my PR #412

@hrishin hrishin force-pushed the maven-it-fix branch 11 times, most recently from 255d559 to 56264e1 Compare August 14, 2018 07:25
@hrishin
Copy link
Member Author

hrishin commented Aug 14, 2018

Thanks @bartoszmajsak.
https://github.com/hrishin/pr-test-vertex/pull/4/files#diff-58231b16fdee45a03a4ee3cf94a9f2c3R15 is what we need to reflect for all the booster projects and whoever using F8-P-L mavenCI with Arquillian tests.

@sthaha @bartoszmajsak could you review this patch?

@@ -61,8 +61,16 @@ def call(body) {
}
}

def testCmd = config.integrationTestCmd
if(config.runTestsInUserNamespace == true && testCmd) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. == true is superfluous and can be deleted
if ( config.runTestsInUserNamespace && testCmd) {
}

nit: please leave a space after keywords

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In if (config.runTestsInUserNamespace), if runTestsInUserNamespace has any value (non empty) then it would eveluate the if block. Hence to force boolean contract it's comparing it with true or false.

Does that sounds right?

def testCmd = config.integrationTestCmd
if(config.runTestsInUserNamespace == true && testCmd) {
def nameSpace = utils.getUsersNamespace()
echo "Running the integration tests in the namespace: ${nameSpace}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nameSpace -> namespace
and we don't need "the" in "... in the namespace: ${namespace}"

if(config.runTestsInUserNamespace == true && testCmd) {
def nameSpace = utils.getUsersNamespace()
echo "Running the integration tests in the namespace: ${nameSpace}"
testCmd = testCmd + " -Dnamespace.use.existing=" + nameSpace
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't this be handled in the mavenIntegrationTest instead?

mavenIntegrationTest {
  namespace = "foobar"
}

If user sets a testCmd, wouldn't they expect to run exactly that?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we only talk about using this library in OSIO Build/Jenkins - then yes. Otherwise - why should I need that?

This patch allows the `mavenCI{}` to over maven command in order
to allow `mavenIntegrationTes{}` to pass specific command to execute.
It has added a util method to retrive default test namespace.

Fixes: openshiftio/openshift.io#763
Fixes: openshiftio/openshift.io#3134
Related to: fabric8io#412
@hrishin
Copy link
Member Author

hrishin commented Aug 16, 2018

@sthaha @bartoszmajsak I've updated the patch could you please review it?

Jenkinsfiles : here

Test results:

[INFO] Running io.openshift.booster.OpenShiftIT
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
CubeKubernetesConfiguration: 
  namespace = hshinde

@@ -63,6 +63,7 @@ def call(body) {

stage('Integration Testing') {
mavenIntegrationTest {
integrationTestCmd = config.integrationTestCmd
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should just be called cmd instead of integrationTestCmd since we have already established that the current context is mavenIntegrationTest

@@ -2,35 +2,37 @@
import io.fabric8.Utils

def call(body) {
Copy link
Contributor

@sthaha sthaha Aug 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about this

def mvnTestCommandForConfig(config) {
    def utils = new Utils()

    def envName = config.environment
    def kubeNS = "-Dfabric8.environment=${envName}"

    if (envName) {
        try {
            def ns = utils.environmentNamespace(envName)
            if (ns) {
                kubeNS = "-Dnamespace.use.existing=${ns}"
                echo "Running the integration tests in namespace : ${ns}"
            }
        } catch (e) {
            echo "ERROR: failed to find the environment namespace for ${envName} due to ${e}"
            e.printStackTrace()
        }
    }

    return "mvn \
      org.apache.maven.plugins:maven-failsafe-plugin:integration-test \
      -P openshift-it ${kubeNS} \
      -Dit.test=${config.itestPattern} \
      -DfailIfNoTests=${config.failIfNoTests}  \
      org.apache.maven.plugins:maven-failsafe-plugin:verify"
}

def call(body) {
    def utils = new Utils()

    if (utils.isDisabledITests()) {
        echo "WARNING: Integration tests DISABLED for these pipelines!"
        return
    }

    // evaluate the body block, and collect configuration into the object
    def config = [:]
    body.resolveStrategy = Closure.DELEGATE_FIRST
    body.delegate = config
    body()


    // execute integration test command if it is explicitly set and exit
    def cmd = "" config.cmd ?: mvnTestCommandForConfig(config)
    sh cmd
    junitResults(body);
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't tested this but the idea is quite simple ... The call either makes the cmd to run or uses the cmd passed to it and it is quite clear IMHO

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks more readable +1

Copy link
Contributor

@sthaha sthaha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please consider the implementation that I pasted?

@hrishin
Copy link
Member Author

hrishin commented Aug 24, 2018

@sthaha I have updated the patch. Could you please review it?

Jenkinsfile:
same as above

Test results:
https://pastebin.com/uQZQCXm9

ReadMe.md Outdated
- pass `cmd` parameter to override the `mvn` command to execute in integration test
```groovy
mavenIntegrationTest {
integrationTestCmd = 'mvn -P openshift-it org.apache.maven.plugins:maven-failsafe-plugin:verify'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this would now be cmd - right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 @sthaha.
Fixed it.

- extracted default maven command code to separate method
- changed variable names
- updated readme
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants