Test Suite Creation

Initial Selection - Excluded Test Cases from the Selected Team

Our test case selection strategy for combining the test suite involved selecting the team with the highest number of test cases first. The test cases from this team were typically included in the combined test suite to serve as a basis. However, some test cases were excluded, as they were either duplicates of an already existing test case in the test suite (and thus redundant) or were identified as incorrect due to issues such as misconfiguration, unexpected crashes, unexpected failures, or false passes.

The following test cases from the initially selected team were excluded:

• ts-price-service (Team 1):

  We identified the test cases bodyVarPricerateInvalidTestValueTooLow() and bodyVarPricerateTestValueIsNegative() in the component test class DeletePriceTest as equivalent, as the former test case already includes a negative number as input, which is exactly what the latter test case is designed to test.

• ts-price-service (Team 1):

  We identified the test cases validTestUpdatesObjectCorrectly() and validTestsSameObject() in the component test class PutPriceTest as equivalent. Both test cases verify whether an object can be successfully updated. The difference is that in the latter test case, the input object is identical to the object already in the database, effectively updating the object with values it previously had.

• ts-train-service (Team 1):

  We identified the test cases invalidTestNonCorrectFormatId(), validTestWrongCharacters(), and invalidTestNonExistingId() as equivalent. The first test case uses an ID with an "invalid" format (e.g., "abc123"), while the second test uses an ID containing "special" characters (e.g., "!@#"). However, as the input of the endpoint is a string without any constraints, it accepts any string. In both cases, the service behaves identically by treating the input as non-existent, a behavior that has already been verified by the test invalidTestNonExistingId()

• ts-price-service (Team 1):

  We identified the test cases validTestGetZeroObjects() and validTestUnusualId() in the component test class GetPriceForRouteTest as equivalent. The former test case tests the behavior of the service with random input string that does not exist in the database, while the latter test case also uses random input string with special characters that does not exist in the database. As the service takes a string as input without any restrictions, both inputs are treated equally by the service. Therefore, we consolidated these test cases into a single test case that verifies the service's behavior when the input string does not exist in the database.

• ts-price-service (Team 1):

  The component test class DeletePriceTest class contains multiple test cases in which the ID matches the PriceConfig to be deleted, but the other values of the PriceConfig object vary across the test cases. As only the ID is relevant for deleting the PriceConfig object, we consolidated these test cases into a single test case that verifies whether a PriceConfig object can be successfully deleted.

• ts-preserve-service (Team 1):

  The integration test class PostPreserveTest includes multiple test cases that verify the behavior of the service when other services are unavailable (e.g., defectTestUnavailableNotificationService(), defectTestUnavailableConsignService(), defectTestUnavailableFoodService(), defectTestUnavailableAssuranceService()). However, the service's behavior does not depend on which specific service is unavailable. Therefore, we consolidated these test cases into a single test case that verifies the service's behavior when its services are generally unavailable.

• ts-preserve-other-service (Team 1):

  The integration test class PostPreserveOtherTest includes multiple test cases that verify the behavior of the service when other services are unavailable (e.g., defectTestUnavailableNotificationService(), defectTestUnavailableConsignService(), defectTestUnavailableFoodService(), defectTestUnavailableAssuranceService()). However, the service's behavior does not depend on which specific service is unavailable. Therefore, we consolidated these test cases into a single test case that verifies the service's behavior when its services are generally unavailable.

• General (Team 3):

  Similar to the point above, team 3 included multiple test cases that verify the behavior of the service when other services are unavailable for all integration test classes. As the service again does not depend on which specific service is unavailable, we consolidated these test cases into a single test case that varifies the service's behavior when its services are generally unavailable

• ts-payment-service (Team 1):

  We identified the test cases invalidTestMultipleObjects() and invalidTestDuplicateObject() in the component test class PostPaymentTest as equivalent. Both test cases verify how the service behaves when multiple objects are provided as input in the request body (as a JSON array). The only difference is that, in the former test case, the second object in the JSON array is different. However, this variation does not alter the service's behavior compared to the latter test case. Therefore, we consolidated these test cases into a single test case that verifies the service's behavior when multiple objects are provided as JSON array.

• ts-inside-payment-service (Team 1):

  The test case invalidTestNonCorrectFormatId() in the component test class GetInsidePayServiceInsidePaymentDrawbackUserIdMoneyTest performs the following request: mockMvc.perform(get("/api/v1/inside_pay_service/inside_payment/drawback/{userId}/{money}", null, "1") to test the behavior when the userId is invalid. However, this approach is ineffective, as the resulting request URL would be /api/v1/inside_pay_service/inside_payment/drawback//1, which is immediately rejected due as the URL is malformed.

• ts-config-service (Team 1):

  We identified the test cases validTestCorrectObject() and validTestUpdatesObjectCorrectly() in the component test class PutConfigsTest as equivalent. Both test cases are identical in their arrangement and act phases, with the only difference being in their assertions. We combined these different assertions into a single test case to eliminate redundancy.

• ts-travel-service (Team 1):

  We identified the test cases validTestGetAllObjects() and validTestGetAllObjectsDetail() in the integration test class GetAdminTripTest as equivalent. The former test case verifies only that the returned object is a non-empty list, while the latter test case also verifies the content of the returned object. Since the latter test case covers what the former test case covers, and is additionally more detailed, we can omit the former test case.

• ts-travel-service (Team 1):

  We identified the test cases validTestObject() and invalidTestTripResponseNull() in the integration test class PostTripsLeftTest as equivalent. The latter test case verifies the behavior of the service when the response is null. However, the response of the former test case is also null, making both tests equivalent and one of them redundant.

• ts-travel2-service (Team 1):

  We identified the test cases validTestGetAllObjectsSuccess() and validTestGetAllObjectsCorrectly() in the integration test class GetAdminTripTravel2ServiceTest as equivalent. The former test case verifies only that the returned object is a non-empty list, while the latter test case also verifies the content of the returned object. Since the latter test case covers what the former test case covers, and is additionally more detailed, we can omit the former test case.

Breakdown of Test Cases per Service

The distribution of test cases across the system is detailed in the following table, which includes counts for all component and integration tests, as well as the amount of failing tests, unstable tests, fixed previously failing tests, and AuthTests.

Service	# ALL Component Tests	# FAILING Component Tests	# ALL Integration Tests	# FAILING Integration Tests	# UNSTABLE Tests	# FIXED PREVIOUSLY FAILING Tests	# AuthTests
ts-admin-basic-info-service	97
ts-admin-order-service	43	4					12
ts-admin-route-service	22	3					9
ts-admin-travel-service	33	1
ts-admin-user-service	19
ts-assurance-service	42	7
ts-auth-service	30	3	5
ts-basic-service	14		6	1
ts-cancel-service	25	2	15
ts-config-service	43	6					15
ts-consign-price-service	20	6
ts-consign-service	27		7
ts-contacts-service	35
ts-execute-service	18	2	8				6
ts-food-map-service	21	3
ts-food-service	33	1	4
ts-inside-payment-service	34	2	9	3
ts-notification-service	36	4					12
ts-order-other-service	83	10	4	1		6
ts-order-service	130	16	4	1		6	47
ts-payment-service	22
ts-preserve-other-service	15	1	14	2		1
ts-preserve-service	15	1	14	2		1
ts-price-service	49
ts-rebook-service	24	3	23	5
ts-route-plan-service	28		9				9
ts-route-service	37	1
ts-seat-service	20	1	12		1
ts-security-service	43	6	7				15
ts-station-service	56	6
ts-ticketinfo-service	11
ts-train-service	24	3
ts-travel-plan-service	24		20	4
ts-travel-service	70	5	20	4
ts-travel2-service	70	4	20	3
ts-user-service	45	1	9	1		10	15
ts-verification-code-service	7	1
SUM	1365	103	210	27	1	24	140