Skip to content

Commit e3308d4

Browse files
raul-arabaolazaraul-arabaolaza
authored
Merge pull request #495 from gbhat618/integration-test-updates
Update integration tests
2 parents 8aeebbd + 0f341b8 commit e3308d4

23 files changed

+513
-246
lines changed

‎CONTRIBUTING.md‎

Lines changed: 69 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -54,45 +54,78 @@ An essential part of getting your change through is to make sure all existing te
5454
* Make sure you are at the directory where pom.xml is located.
5555

5656
##### Unit Tests
57-
* Run the following:
58-
59-
```
60-
mvn test
61-
```
57+
* Run the following:
58+
```
59+
mvn test
60+
```
6261
6362
##### Integration Tests
64-
* By default, the integration tests are not executed. In case you are interested in executing
65-
them, disable the `skipITs` property.
66-
* The following environment variables are required to run the integration tests. 5, 6, and 7 are
67-
only required when running a windows integration test.
68-
69-
1. GOOGLE_PROJECT_ID
70-
1. GOOGLE_CREDENTIALS
71-
1. GOOGLE_REGION
72-
1. GOOGLE_ZONE
73-
1. GOOGLE_BOOT_DISK_PROJECT_ID
74-
1. GOOGLE_BOOT_DISK_IMAGE_NAME
75-
1. GOOGLE_JENKINS_PASSWORD
76-
1. GOOGLE_SA_NAME
77-
78-
* Run the following:
79-
```
80-
mvn verify -DskipITs=false
81-
```
63+
64+
Integration tests provision actual instances in a GCP project, run pipeline, take snapshot etc.
65+
Therefore, they are disabled in the CI and expected to be executed by contributors in their laptop itself.
66+
67+
Reasons for disabling integration test in CI,
68+
* getting provisioning GCP infra is not possible
69+
* even if we did get a GCP infra setup in the CI, it is risky to expose that, as someone can abuse the CI.
70+
71+
By default, integration tests are skipped from the maven goals, need to enable using the `skipITs` property.
72+
73+
Steps to execute integration test
74+
* Prepare VM images
75+
(ideally we should automate this see idea [here](https://github.com/jenkinsci/google-compute-engine-plugin/pull/492#discussion_r1892705637))
76+
The jenkins agent images need to have java installed. We have a packer script to create the image and upload to your configured GCP project.
77+
The scripts are located in [testimages/linux](./testimages/linux)
78+
Navigate to the directory and execute,
79+
```bash
80+
bash setup-gce-image.sh
81+
```
82+
* The above agent image contains the `java` command available on the PATH; which the plugin uses by default for launching the agent.
83+
This plugin also supports configuring custom path for java executable, and we have an integration test for that `ComputeEngineCloudNonStandardJavaIT`.
84+
If you would like to execute this test, please create an image with `java` command not being on the PATH, but at a custom path `/usr/bin/non-standard-java`.
85+
To create a non-standard java image, execute,
86+
```bash
87+
bash setup-gce-image.sh non-standard-java
88+
```
89+
If you want to delete the images or recreate them, use the arguments `--recreate` or `--delete`.
90+
91+
* Create a service account with relevant access - See [Refer to IAM Credentials](Home.md#iam-credentials)
92+
93+
* Export these mandatory environment variable
94+
```bash
95+
export GOOGLE_PROJECT_ID=your-project-id
96+
export GOOGLE_CREDENTIALS=/path/to/sa-key.json
97+
export GOOGLE_REGION=us-central1
98+
export GOOGLE_ZONE=us-central1-a
99+
export GOOGLE_SA_NAME=jenkins-agent-sa
100+
```
101+
* Run the integration tests as,
102+
* Run all the tests
103+
```bash
104+
mvn verify -DskipITs=false
105+
```
106+
* Run a specific test class
107+
```bash
108+
mvn clean test -Dtest=ComputeEngineCloudRestartPreemptedIT
109+
```
110+
* Run a specific test method
111+
```bash
112+
mvn clean test -Dtest=ComputeEngineCloudRestartPreemptedIT#testIfNodeWasPreempted
113+
```
114+
You can also debug the tests with surefire by passing `-Dmaven.surefire.debug=true` and in your IDE connect to remote debug port `8000`.
82115

83116
###### Windows Integration Test
84117
* By default, the integration tests only use linux based agents for testing. If you make a
85118
windows-related change, or otherwise want to test that a change still works for windows agents,
86-
run the tests with the flag `-Dit.windows=true` like this:
87-
```bash
88-
mvn verify -Dit.windows=true
89-
```
90-
91-
Make sure you have these extra environment variables configured:
92-
* GOOGLE_BOOT_DISK_PROJECT_ID will be the same as your project id.
93-
* GOOGLE_BOOT_DISK_IMAGE_NAME will be the name of the image you created using packer in Google
94-
cloud console.
95-
* GOOGLE_JENKINS_PASSWORD will be the password you set when creating the image with packer, used
96-
for password based ssh authentication.
97-
* More information on building your baseline windows image can be found [here](WINDOWS.md)
98-
and an example powershell script for setup can be found [here](windows-it-install.ps1).
119+
run the tests with the flag `-Dit.windows=true` like this:
120+
```bash
121+
mvn verify -Dit.windows=true
122+
```
123+
* You need to prepare the windows image before running the tests.
124+
* More information on building your baseline windows image can be found [here](WINDOWS.md)
125+
and an example powershell script for setup can be found [here](windows-it-install.ps1).
126+
* In addition to the environment variables mentioned in the previous section, also export these variables too,
127+
```bash
128+
export GOOGLE_BOOT_DISK_PROJECT_ID=your-project-id # will be the same as your project id
129+
export GOOGLE_BOOT_DISK_IMAGE_NAME=windows-image-name # will be the name of the image you created using packer in Google cloud console
130+
export GOOGLE_JENKINS_PASSWORD=password # will be the password you set when creating the image with packer, used for password based ssh authentication.
131+
```

‎pom.xml‎

Lines changed: 26 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,6 @@
6666
<changelist>999999-SNAPSHOT</changelist>
6767
<gitHubRepo>jenkinsci/${project.artifactId}-plugin</gitHubRepo>
6868
<jenkins.version>2.452.3</jenkins.version>
69-
<jenkinsRuleTimeout>0</jenkinsRuleTimeout>
7069
<skip.surefire.tests>${skipTests}</skip.surefire.tests>
7170
<skipITs>true</skipITs>
7271
<it.windows>false</it.windows>
@@ -83,6 +82,7 @@
8382
<lombok.version>1.18.30</lombok.version>
8483
<delombok.output>${project.build.directory}/delombok</delombok.output>
8584
<spotless.check.skip>false</spotless.check.skip>
85+
<jenkins.test.timeout>600</jenkins.test.timeout>
8686
</properties>
8787

8888
<dependencyManagement>
@@ -269,7 +269,27 @@
269269
<version>${powershell.version}</version>
270270
<scope>test</scope>
271271
</dependency>
272-
<!-- Test Dependencies -->
272+
<!-- https://www.jenkins.io/doc/developer/testing/#working-with-pipeline -->
273+
<dependency>
274+
<groupId>org.jenkins-ci.plugins.workflow</groupId>
275+
<artifactId>workflow-basic-steps</artifactId>
276+
<scope>test</scope>
277+
</dependency>
278+
<dependency>
279+
<groupId>org.jenkins-ci.plugins.workflow</groupId>
280+
<artifactId>workflow-cps</artifactId>
281+
<scope>test</scope>
282+
</dependency>
283+
<dependency>
284+
<groupId>org.jenkins-ci.plugins.workflow</groupId>
285+
<artifactId>workflow-durable-task-step</artifactId>
286+
<scope>test</scope>
287+
</dependency>
288+
<dependency>
289+
<groupId>org.jenkins-ci.plugins.workflow</groupId>
290+
<artifactId>workflow-job</artifactId>
291+
<scope>test</scope>
292+
</dependency>
273293
<dependency>
274294
<groupId>org.mockito</groupId>
275295
<artifactId>mockito-core</artifactId>
@@ -355,11 +375,11 @@
355375
<hudson.agents.NodeProvisioner.MARGIN0>1.0</hudson.agents.NodeProvisioner.MARGIN0>
356376
<hudson.remoting.Launcher.pingIntervalSec>-1</hudson.remoting.Launcher.pingIntervalSec>
357377
<com.google.jenkins.plugins.computeengine.integration.ITUtil.windows>${it.windows}</com.google.jenkins.plugins.computeengine.integration.ITUtil.windows>
378+
<jenkins.test.timeout>${jenkins.test.timeout}</jenkins.test.timeout>
358379
</systemPropertyVariables>
359380
<excludes>
360381
<exclude>**/ITUtil.java</exclude>
361382
</excludes>
362-
<argLine>-Djenkins.test.timeout=${jenkinsRuleTimeout}</argLine>
363383
<runOrder>${it.runOrder}</runOrder>
364384
<reuseForks>false</reuseForks>
365385
</configuration>
@@ -378,6 +398,9 @@
378398
<skipTests>${skip.surefire.tests}</skipTests>
379399
<runOrder>balanced</runOrder>
380400
<reuseForks>false</reuseForks>
401+
<systemPropertyVariables>
402+
<jenkins.test.timeout>${jenkins.test.timeout}</jenkins.test.timeout>
403+
</systemPropertyVariables>
381404
</configuration>
382405
</plugin>
383406
<plugin>

‎src/test/java/com/google/jenkins/plugins/computeengine/integration/ComputeClientIT.java‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,9 +36,9 @@
3636

3737
/** Integration test suite for {@link ComputeClient}. */
3838
public class ComputeClientIT {
39-
private static Logger log = Logger.getLogger(ComputeClientIT.class.getName());
39+
private static final Logger log = Logger.getLogger(ComputeClientIT.class.getName());
4040

41-
private static Map<String, String> label = getLabel(ComputeClientIT.class);
41+
private static final Map<String, String> label = getLabel(ComputeClientIT.class);
4242
private static ComputeClient client;
4343

4444
@ClassRule
@@ -58,7 +58,7 @@ public static void teardown() throws IOException {
5858

5959
@Test
6060
public void testGetImage() throws Exception {
61-
Image image = client.getImage("debian-cloud", "debian-9-stretch-v20180820");
61+
Image image = client.getImage("debian-cloud", "debian-12-bookworm-v20241210");
6262
assertNotNull(image);
6363
assertEquals("READY", image.getStatus());
6464
}

‎src/test/java/com/google/jenkins/plugins/computeengine/integration/ComputeEngineCloud1WorkerCreatedFor2ExecutorsIT.java‎

Lines changed: 20 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -20,22 +20,24 @@
2020
import static com.google.jenkins.plugins.computeengine.integration.ITUtil.NULL_TEMPLATE;
2121
import static com.google.jenkins.plugins.computeengine.integration.ITUtil.PROJECT_ID;
2222
import static com.google.jenkins.plugins.computeengine.integration.ITUtil.TEST_TIMEOUT_MULTIPLIER;
23-
import static com.google.jenkins.plugins.computeengine.integration.ITUtil.ZONE;
2423
import static com.google.jenkins.plugins.computeengine.integration.ITUtil.getLabel;
2524
import static com.google.jenkins.plugins.computeengine.integration.ITUtil.initClient;
2625
import static com.google.jenkins.plugins.computeengine.integration.ITUtil.initCloud;
2726
import static com.google.jenkins.plugins.computeengine.integration.ITUtil.initCredentials;
2827
import static com.google.jenkins.plugins.computeengine.integration.ITUtil.instanceConfigurationBuilder;
2928
import static com.google.jenkins.plugins.computeengine.integration.ITUtil.teardownResources;
29+
import static org.awaitility.Awaitility.await;
30+
import static org.hamcrest.Matchers.is;
3031
import static org.junit.Assert.assertEquals;
3132

32-
import com.google.api.services.compute.model.Instance;
3333
import com.google.cloud.graphite.platforms.plugin.client.ComputeClient;
3434
import com.google.common.collect.ImmutableList;
3535
import com.google.jenkins.plugins.computeengine.ComputeEngineCloud;
3636
import hudson.model.labels.LabelAtom;
3737
import hudson.slaves.NodeProvisioner.PlannedNode;
3838
import java.io.IOException;
39+
import java.time.Duration;
40+
import java.util.ArrayList;
3941
import java.util.Collection;
4042
import java.util.Map;
4143
import java.util.concurrent.TimeUnit;
@@ -52,20 +54,18 @@
5254
* when configured to support multiple executors if the second executor is not required.
5355
*/
5456
public class ComputeEngineCloud1WorkerCreatedFor2ExecutorsIT {
55-
private static Logger log = Logger.getLogger(ComputeEngineCloud1WorkerCreatedFor2ExecutorsIT.class.getName());
57+
private static final Logger log = Logger.getLogger(ComputeEngineCloud1WorkerCreatedFor2ExecutorsIT.class.getName());
5658

5759
private static final String MULTIPLE_NUM_EXECUTORS = "2";
5860

5961
@ClassRule
60-
public static Timeout timeout = new Timeout(5 * TEST_TIMEOUT_MULTIPLIER, TimeUnit.MINUTES);
62+
public static Timeout timeout = new Timeout(5L * TEST_TIMEOUT_MULTIPLIER, TimeUnit.MINUTES);
6163

6264
@ClassRule
6365
public static JenkinsRule jenkinsRule = new JenkinsRule();
6466

6567
private static ComputeClient client;
66-
private static Map<String, String> label = getLabel(ComputeEngineCloud1WorkerCreatedFor2ExecutorsIT.class);
67-
private static Collection<PlannedNode> planned;
68-
private static Instance instance;
68+
private static final Map<String, String> label = getLabel(ComputeEngineCloud1WorkerCreatedFor2ExecutorsIT.class);
6969

7070
@BeforeClass
7171
public static void init() throws Exception {
@@ -83,10 +83,8 @@ public static void init() throws Exception {
8383
.googleLabels(label)
8484
.build()));
8585

86-
planned = cloud.provision(new LabelAtom(LABEL), 2);
86+
Collection<PlannedNode> planned = cloud.provision(new LabelAtom(LABEL), 2);
8787
planned.iterator().next().future.get();
88-
89-
instance = client.getInstance(PROJECT_ID, ZONE, planned.iterator().next().displayName);
9088
}
9189

9290
@AfterClass
@@ -95,12 +93,18 @@ public static void teardown() throws IOException {
9593
}
9694

9795
@Test
98-
public void test1WorkerCreatedFor2ExecutorsStatusRunning() {
99-
assertEquals("RUNNING", instance.getStatus());
100-
}
96+
public void test1WorkerCreatedFor2ExecutorsStatusRunning() throws IOException {
97+
// assert on jenkins side
98+
assertEquals(1, jenkinsRule.jenkins.getNodes().size());
99+
assertEquals(2, jenkinsRule.jenkins.getNodes().get(0).getNumExecutors());
101100

102-
@Test
103-
public void test1WorkerCreatedFor2ExecutorsOnly1Planned() {
104-
assertEquals(1, planned.size());
101+
// assert on gcp side
102+
assertEquals(1, new ArrayList<>(client.listInstancesWithLabel(PROJECT_ID, label)).size());
103+
await().timeout(Duration.ofMinutes(2))
104+
.until(
105+
() -> client.listInstancesWithLabel(PROJECT_ID, label)
106+
.get(0)
107+
.getStatus(),
108+
is("RUNNING"));
105109
}
106110
}

0 commit comments

Comments
 (0)