Fault Identification
We color-coded mentions to quickly identify what kind of category the quote belongs to:
🔴 No Fault
🔵 Unconventional design choice
🟣 Special Issue: triggered fallback response on Error
🟢 Fault
Mentions are marked in quotes. If needed, an additional explanation was provided below the quote to further explain the decision for identification or to give a clearer description for a fault.
Component Testing:
-
🔴 POST /api/v1/admintravelservice/admintravel
”Tests the scenario where a duplicate travelInfo object is added to ts-travel-service. Mocks a response indicating that the trip already exists and verifies the expected response indicating that the trip already exists and verifies the expected response status and message. The test fails, because the response of ts-travel-service always has status 1”
→ This test was written with consideration of what the ts-travel-service does. If we do not know that the travel service has an issue and therefore always returns status 1, all tests for the admintravel service would pass, because this endpoint was implemented correctly. Therefore, this is a NON-FAULT for the admintravel service specifically.
-
🟢 [F2] POST /api/v1/auth
”Test case for an invalid request where the password is too short. The test expects a UserOperationException to be thrown, because shortest expected length for password is 6 TEST FAILS Response should contain message “Password must contain at least 6 characters.” […] But the actual response is status created, no exception and message success. Do not know why the user actually gets created, because it shouldn’t”
→ The system encodes the provided password using
.password(passwordEncoder.encode(dto.getPassword()))and after, checks the length of the now encoded password. Since the encoded password always meets the criteria of being at least 6 characters long, the error of having a password that is too short can never be reached -
🔵 [D6] PostContactServiceContactsAdmin
”The test is designed to verify that the endpoint for creating a new contacts correctly handles the case when there already exists a contact with the same ID. it ensures that the endpoint returns a response with the appropriate message and the contact. The test fails because the controller creates a new ID for the contact, therefore the old contact ID is not found and the contact is created with the new ID”
→ In the ts-contact-service, a method verifies whether a contact ID exists before creating and saving a new contact object to the database. However, the controller method responsible for invoking this check generates a new random ID beforehand. As a result, the service consistently successfully saves a new contact object to the database, allowing the creation of multiple contact objects with the same information but different IDs. As the system lacks formal specification, we do not know whether the system is behaving as it is designed to. It could be argued that this specific implementation was intentional, because the system consistently creates contact objects with a unique ID in the database. Therefore, to grant the original developer the benefit of the doubt, this implementation was deemed as intentional. However, this issue is a poor design decision for two reasons: (a) the controller prematurely assigns IDs, which should be the responsibility of the service layer, and (b) the duplicate check on ID is redundant when a unique ID is generated beforehand.
-
🟢 [F4a] GetFoodMapServiceTrainFoodsTripId
”[…] Test fails because listTrainFoodByTripID() function in FoodMapServiceImpl only checks if train foods is null, not if the list is empty.”
-
🟢 [F4a]
”[…] Test fails because getFoodStoresByStationIds() function in FoodMapServiceImpl only checks if train foods is null, not if the list is empty.”
-
🟢 [F4b] GetInsidePayServiceInsidePaymentUserIdMoney
”[…] Test fails because function logic of addMoney() in InsidePaymentServiceImpl is incorrect. Return value for addMoneyRepository.findByUserId(userId) check for != null but not if list is empty”
-
🟢 [F4b] GetInsidePayServiceInsidePaymentDrawbackUserIdMoney
”[…] Test fails because function logic of drawback() in InsidePaymentServiceImpl is incorrect. Return value for addMoneyRepository.findByUserId(userId) check for != null but not if list is empty”
-
🟢 [F4f] GetOrderOtherServiceOrderOther
”[…] Test fails because getAllOrders() function in OrderOtherServiceImpl only checks if orders are null but not if the list is empty”
-
🟢 [F25c] PostPreserveOther
”[…] but because of a wrong implementation for the economy class (comparison of number of seats with 3 instead of seatType as well as only checking number of seats in comfort class) does not. That is why this test fails.”
→ Instead of:
it should be:
-
🟢 [F25b] PostPreserve
”[…] but because of a wrong implementation for the economy class (comparison of number of seats with 3 instead of seatType as well as only checking number of seats in comfort class) does not. That is why this test fails.”
→ Instead of:
it should be:
-
🟢 [F25a] PostRebook
”[…] but because of a wrong implementation for the economy class (comparison of number of seats with 3 instead of seatType as well as only checking number of seats in comfort class) does not. That is why this test fails.”
→ Instead of:
it should be:
-
🟢 [F21] PostSeats
”The seat type can either be 2 or 3. So here we test the equivalence class, where the value is out of range. But as it is only compared with 2 and every other value is automatically assumed to be 3, the request will be executed as normal. Another team member mentioned, that this test fails for them, because of a NullPointerException, but it works for me.”
→ This test can sometimes cause a NullPointerException, and sometimes it does not. This is because of the setup done before executing the endpoint itself. We create tickets and randomly assign seatsNumbers to it. The service itself has a while loop
Which checks if the current seat is contained as a seatNumber. If yes, it will randomly reassign seat given a range and try again. If the seatNumbers of the tickets happen to contain all seatNumbers that are within the range of
rand.nextInt(range) + 1the service will never be able to exit this while loop, eventually crash and trigger the fallback response with null values. This is how the NullPointerException can happen. Although the trigger of the NullPointerException is because of a misconfiguration of a test, the fact that we are able to get into an infinite while-loop can be considered as a mistake and could be categorized as a fault. -
🔴 PutTrain
”The PUT endpoint is not implemented correctly. The PUT endpoint should be used to update an existing object. It creates a new object instead. This is why the test cases are not working”
→ The Spring Data JPA will perform an update if an entity with the same ID already exists. Therefore, it is fine to create a new object and save it, as long as it has the same ID, which is the case here. Therefore, this is a NON-FAULT.
-
🟢 [F5a] DeleteTrain
”The implementation of the Endpoint is not correct, this is why this test fails.”
→ Null check instead of Optional.empty check. For this endpoint specifically, it will cause
repository.deleteById(id);to always be executed and the delete to always be successful, even when there was nothing to be deleted. -
🟢 [F6] GetTravelServiceRoutesTripId
”The test fails because the getRouteByTripId() function in TravelServiceImpl checks for route != null but the return value from the ts-route-service sets route attributes to null Actual return: Route{id='null', stations=null, distances=null, startStationId='null', terminalStationId='null'}”
→ when route service in incapable of finding a route, it fails and will return null. The travel service however will configure an empty Route, which which will lead to a check whether the list of returned routes is empty to always be false:
Integration Testing:
-
🟢 [F13] PostBasic
”TEST FAILS When trainTypeId and so trainType does not exist (=null), trainTypeString is set to "". […] Because the trainTypeString is "", nothing is put after the last "/", and so the response status is 404, Not Found and therefore the test fails.”
-
🟣 [S1] PostConsignServiceConsigns
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟣 [S1] PutConsignServiceConsigns
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟣 [S1] GetExecuteServiceCollectedOrderId
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟣 [S1] GetExecuteServiceExecuteOrderId
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟣 [S1] GetFoodServiceFoodsDateStartStationEndStationTripId
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🔵 [D4] PostInsidePayServiceInsidePayment
”for the case that the price is negative It ensures that the endpoint returns a response with the appropriate message and no content. The test fails because the implementation allows negative values as price.”
→ For this specific endpoint, userID, and especially price is irrelevant for the functionality of this endpoint. The insidepayment service calls the order service with the orderID from the request body, and the found order itself has a price, which will be used instead of the provided price of the payment info, which is reasonable. The fact that you could however provide values which wont be utilized by the service is a bad design choice
-
🟣 [S1] PostInsidePayServiceInsidePayment
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟢 [F14] PostInsidePayServiceInsidePaymentDifference
”[…] The test fails because of incorrect service implementation, so that ts-payment-service is never called […]”
→ BigDecimal library is used incorrectly. This causes a service dependent on an if-condition to be never true
-
🟣 [S1] PostCheapestRoute
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟣 [S1] PostQuickestRoute
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟣 [S1] PostMinStopStations
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟣 [S1] PostMinStopStations
”test fails because the ts-route-service returns null as data when no route is found and the function is attempting to call data.size() which leads to a NullPointerException.”
→ The NullPointerException will trigger a fallback response. The issue with the fallback response is that it remains unclear whether developers deliberately omitted preventative measures (e.g. null-check) in favor of relying on the fallback (which would be considered intended behavior), or not.
-
🟣 [S1] PostLeftTickets
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟣 [S1] PostSeats
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟢 [F13] PostTicketInfo
”TEST FAILS When trainTypeId and so trainType does not exist (=null), trainTypeString is set to "". […] Because the trainTypeString is "", nothing is put after the last "/", and so the response status is 404, Not Found and therefore the test fails.”
-
🟣 [S1] PostTravelPlanMinStation
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟣 [S1] PostTravelPlanQuickest
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟣 [S1] PostTravelPlanCheaperst
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟣 [S1] PostTravelPlanTransferResult
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟢 [F6] GetRoutesTripIdTravel2Service
”The test fails because the getRouteByTripId() function in TravelServiceImpl checks for route != null but the return value from the ts-route-service sets route attributes to null Actual return: Route{id='null', stations=null, distances=null, startStationId='null', terminalStationId='null'}”
→ when route service in incapable of finding a route, it fails and will return null. The travel service however will configure an empty Route, which which will lead to a check whether the list of returned routes is empty to always be false:
-
🟣 [S1] PostUsersRegister
”If a dependent service is unavailable, the application should handle this gracefully without crashing or providing misleading information. The test is designed as part of defect-based testing to simulate real-world scenarios where service dependencies may fail. The test fails because the implementation returns a 200 status with null values when the service is unavailable.”
→ The system does indeed not crash in such a case. The information given to the user is however not useful at all, which could both be argued as a fault and a bad design choice of the fallback response itself. Therefore, SPECIAL CASE.
-
🟢 [F24] PostRebookDifferenceTest
”Because the deletion of the order at then end is doing a request to the endpoint with a wrong POST instead of DELETE, this test fails”
-
🟢 [F4e] PostRebookTest
”This test fails because of an implementation error in insidePaymentService where an empty list is compared to null for a non-existing id. As such the request is always a success, which it should not be.”
-
🟢 [F20a] (F20b for orderOther) PostOrderServiceOrderRefreshTest
”With the station ids being invalid the ts-station service returns
*Response(status=0, msg=No stationNamelist according to stationIdList, data=[]). But since the implementation does not check the return value, there is an IndexOutOfBoundsException.”*
Component Testing:
-
🟢 [F7] /consignpriceservice/consignprice
”This endpoint does not work properly since it saves a second consign price with index 0. This makes all endpoints fail afterwards because two instance with index 0 exist from now on.”
→
repository.findByIndex(0)returns a ConsignPrice Object, not a list. This does not work however when there are more than one ConsignPrice instance with index 0 in the database. -
🟢 [F8c] POST: “/api/v1/routeservice/routes”
”RCAM6 - RCAM9 intentionally fail because there are no validity checks for parameters”
→ The service assumes that the object in the requestBody is fully defined and uses that object without performing null-checks. This will lead to a NullPointerException that is unhandled if the values within the provided object are accessed
-
🔴 GET: “/api/v1/routeservice/routes/{startId}/{terminalId}”
”RQST5 and RQST6 intentionally fail, as a client error should be thrown if the start or end terminal are empty strings → invalid input”
→ An IllegalArgumentException is expected returned if the user does not provide a startId or terminalId or just provide empty strings
Integration Testing:
-
🟢 [F13] /basic/trip
”Test Case 03 fails intentionally, because Basic Service Implementation is incorrect and calls Price Service with no route Id after recognizing that route does not exist”
→ A request to an endpoint is made that does not exist, as the basic service attempts to make a request to the price service without providing a PathVariable
-
🟢 [F21] /rebookservice/rebook/difference
”Since the travel service “trip_detail” endpoint was unstable when tested this call is skipped. Also the call to the seat service caused some issues. The communication inside the seat service would be tested and is therefore addressed inside the possible defects. The possible defects regarding the travel service “trip_detail” endpoint look are listed inside the documentation of this endpoint.”
→ The instability mentioned likely refers to F21, which occasionally triggered an infinite while loop
-
🟢 [F9a] orderService and [F9b] orderOtherService (date)
”had to adjust date of order in Init from ts-order-service, as deprecated Order version was originally used that could not be used for comparison”
→ Endpoint takes a Date object as a parameter in paths which will falsely be interpreted as a string. This will always lead to an IllegalArgumentException when attempting to call that endpoint
-
🟢 [F9 & F21] /travelservice/trips/left
”Inside the Travel Service Implementation the call to the order service is commented since this will result in code 400 everytime. Also the seat service is not called since this led to multiple problems. Therefore some of the listed possible defects are only hypothetical tested. The test cases are still very unstable when performed locally, this means they are failing nearly every second time even though nothing was changed.”
→ The 400 roots back to F9, the instability likely refers to F21
-
🟢 [F9 & F21] /travelservice/trip_detail
”Inside the Travel Service Implementation the call to the order service is commented since this will result in code 400 everytime. Also the seat service is not called since this led to multiple problems. Therefore some of the listed possible defects are only hypothetical tested. The test cases are still very unstable when performed locally, this means they are failing nearly every second time even though nothing was changed.”
→ The 400 roots back to F9, the instability likely refers to F21
General Issues in Component Testing:
-
🔴 General
”Services that are using @Annotation for validation of data aren't implementing the validation of spring-boot correctly thus these annotations are ignored and invalid data can be passed into the database. → In Spring Boot, validation annotations (like
@NotNull,@Size, etc.) are used to ensure that the data being processed meets certain criteria. If these annotations are not correctly implemented or configured, the validation will be ignored, allowing invalid data to pass through. Because @Validation orspring-boot-starter-validationdependency is missing”→ spring-boot-starter-validation dependency is not missing and part of pom.xml.
-
🟢 [F15] General (f.e. Configservice [F15a] or Userservice [F15b])
”If during the building of the security chain for spring-boot the function call of permitAll() is the first within the filter chain, then additional security filters are not checked as they are checked by springboot in order from the first to last. Due to the filter setup like "/**" all requests match the first permitAll() filer resulting in no security checks afterwards.”
→ In Spring Security, the order of the filter chain is crucial. The filters are applied in the order they are defined. If
permitAll()is the first filter, it will allow all requests to pass through, and subsequent security filters will be ignored:
-
🟢 [F22] General
”When a request does not have an authentication header and authentication is required to access a resource, then the application should respond with HTTP Status of 401 Unauthorized instead of HTTP Status 403 Forbidden.”
→ To add to that: We expect a 403 if we provide authorization headers, but they are not sufficient for the particular endpoint. If we do no provide any authorization headers, a 401 should be expected if authorization is needed.
-
🟢 [F12] General (f.e. orderservice)
”When a protected route is not defined with a trailing slash, adding a trailing slash to the request URI will bypass the authorization. Example: POST /api/v1/orderservice/order/admin is defined without trailing slash and can be accessed even by unauthenticated users by requesting /api/v1/orderservice/order/admin/”
-
🟢 [F27] General UUID Serialization (f.e. in assurance service [F27a], in security service [F27b], in auth service [F27c], in food service [F27d], in orderOther service [F27e], in order service [F27f]
”If a string, which is not UUID serializable is passed into any endpoint as an entity ID, the system doesn't catch the exception, which is thrown by UUID.fromString(). This results in an uncaught exception, which is handled by spring-boot automatically and thus responds with the status code HTTP 500, and a default error message is returned. This then results in an uncaught Null pointer exception when we are passing this ID into another micro-service without any checks and the default spring-boot error response is returned as the system doesn't expect this response and is trying to parse it as a custom response defined within the system.”
→ Some endpoints, like the ts-assurance-service, attempt to convert the provided String to a UUID object immediately without checking whether this String is UUID serializable, which will throw an unhandled exception.
-
🔴 General
”Not matching success messages. f.e. “Success” and “Success.””
→ Typos will not be considered as faults as they are simply not interesting.
Component Testing:
-
🟢 [F5a] PUT /trainservice/trains
”Null id throws IllegalArgumentException when trying to add to the database. types since Optional is never null.”
-
🟢 [F7] → F7 POST /consignpriceservice/consignprice
”In case the consign price config exists in the repository, update with new UUID causes an error because the application expects a unique consign price with an index of 0.”
-
🟢 [F29] GET /verifycode/generate
”When the user request comes with the captcha cookie, the system generates a new cookie ID instead of using the existing one. When the captcha cookie has no value, the system tries to use the existing one.”
-
🔵 [D6] POST /contactservice/contacts/admin
”User can be created even if another user with the same username/data exists, user roles are allowed access.”
→ In the ts-contact-service, a method verifies whether a contact ID exists before creating and saving a new contact object to the database. However, the controller method responsible for invoking this check generates a new random ID beforehand. As a result, the service consistently successfully saves a new contact object to the database, allowing the creation of multiple contact objects with the same information but different IDs. As the system lacks formal specification, we do not know whether the system is behaving as it is designed to. It could be argued that this specific implementation was intentional, because the system consistently creates contact objects with a unique ID in the database. Therefore, to grant the original developer the benefit of the doubt, this implementation was deemed as intentional. However, this issue is a poor design decision for two reasons: (a) the controller prematurely assigns IDs, which should be the responsibility of the service layer, and (b) the duplicate check on ID is redundant when a unique ID is generated beforehand.
-
🟢 [F2] POST /auth
”user can be created with an empty password and its length can be less than 6 characters.”
→ The system encodes the provided password using
.password(passwordEncoder.encode(dto.getPassword()))and after, checks the length of the now encoded password. Since the encoded password always meets the criteria of being at least 6 characters long, the error of having a password that is too short can never be reached -
🟣 [S1] POST /travelplanservice/transferResult
”If travel data returns null, there is no null check which causes a null exception.”
→ The NullPointerException will trigger a fallback response. The issue with the fallback response is that it remains unclear whether developers deliberately omitted preventative measures (e.g. null-check) in favor of relying on the fallback (which would be considered intended behavior), or not.
-
🟢 [F5a] POST /trainservice/trains
”Null check on Optional class instead of the Optional.isPresent blocks creating new train”
-
🟢 [F5a] DELETE /trainservice/trains/{id}
”Null check on Optional class instead of the Optional.isPresent tries to delete for any id value.”
-
🔵 [D4] POST /consignservice/consigns
”Should not get id as input, since its creation and id is generated already in the logic.”
→ The endpoint still functions as expected. The fact that you could however provide values which wont be utilized by the service is a bad design choice
-
🔵 [D4] POST /assuranceservice/assurances/{assuranceId}/{orderId}/{typeIndex}
”Order ID is not effective in the logic and is not used. Therefore assurances are not modified with the orderId.”
→ As the assurance service seems to function as expected. The orderId is therefore presumably not needed for this particular endpoint. However, having unnecessary variables in the signature of a method is bad practice and should be avoided.
-
🔴 GET /admintravelservice/admintravel
”If travel services return data in a list other than ArrayList exception occurs. The latest service call's status is the status of this endpoint which is unintended behavior.”
→ The travel service is implemented to only ever return an ArrayList. Without altering the source code, it is not possible for the service to return anything else but an ArrayList. Therefore, this is a NON-FAULT.
-
🟢 [F18a] POST /orderservice/order/query
”Travel date filter not implemented correctly”
→ Essentially, the service is trying to do
tempOrder.getTravelDate().after(*null*);which will cause a crash. This is because we compare
instead of
which does not make sense logically.
-
🟢 [F5b] POST /stationservice/stations
”Optional equality checked to null”
-
🔵 [D4] GET /inside_pay_service/inside_payment/drawback/{userId}/{money}
”Invalid orderId input does not return to failure response. Invalid money input does not return to a failure response.”
→ For this specific endpoint, userID, and especially price is irrelevant for the functionality of this endpoint. The insidepayment service calls the order service with the orderID from the request body, and the found order itself has a price, which will be used instead of the provided price of the payment info, which is reasonable. The fact that you could however provide values which wont be utilized by the service is a bad design choice
-
🔵 [D4] GET /inside_pay_service/inside_payment/{userId}/{money}
”Invalid orderId input does not return to failure response. Invalid money input does not return to a failure response.”
→ For this specific endpoint, userID, and especially price is irrelevant for the functionality of this endpoint. The insidepayment service calls the order service with the orderID from the request body, and the found order itself has a price, which will be used instead of the provided price of the payment info, which is reasonable. The fact that you could however provide values which wont be utilized by the service is a bad design choice
-
🟢 [F4a] GET /foodmapservice/trainfoods/{tripId}
”GetTrainFoodOfTrip - Invalid TripId input does not return to failure response.”
→ null check instead of empty list check is responsible for any tripId to always be valid
-
🔵 [D1] DELETE /routeservice/routes/{routeId} - More really bad design that actual fault
”DeleteRouteById - Invalid RouteId input does not return to failure response.”
→ repository.delete is performed before the check whether the object exists, always resulting in “Delete Successful”. This is not a mistake itself, because when no object exists, nothing happens when repository.delete is called. It is however odd to always respond with a successful delete, even when nothing was deleted.
-
🟢 [F4a] POST /foodmapservice/foodstores
”GetFoodStoresByStationIds - Invalid StationIds input does not return to failure response”
→ null check instead of empty list check is responsible for any stationId to always be valid
General Issues in Integration Testing:
-
🟣 [S1] ROUTEPLANSERVICE:
”Bounds checking is missing when merging the trips from ts-travel-service and ts-travel2-service. When they are not equal a size IndexOutOfBounds Exception is thrown which is caught by @Hystrix.”
→ The potential IndexOutOfBounds Exception will be raised here:
Because it is generally possible for travelTrips to be smaller than travel2trips.
Since the IndexOutOfBound will lead to the fallback Response, it is unknown whether the Developer knew that some error like that could happen here or not, because the message of the fallback response is empty. That is why this issue is assigned to SPECIAL CASE.
-
🟣 [S1] ROUTEPLANSERVICE:
”When Routes are found, but the departure Date is in before today's date then the service returns null intentionally. However later on this null is not expected/handled in the code and thus results in a NullPointer Exception which is caught by @Hystrix.”
→ Presumably, what is meant here is this specific line of code in the implementation of the travel services:
Generally, null returned by called services is only rarely checked. It is unknown whether the developer did not implement null-checks by mistake, or whether the developer intentionally did not check for it, as the Exception will be caught by @Hystrix either way. Since the message of the fallback response is non-existent, this issue is assigned to SPECIAL CASE.
-
🟣 [S1] ROUTEPLANSERVICE:
”When Trip Id doesn't start with 'Z', 'T', or 'K' the service encounters NullPointer exception, which crashes the route planning algorithms and returns a response with Http Status 200 and null body due to @Hystrix exception handling. As Routes starting with 'D' or 'G' are retrieved from ts-travel-service, this service can never be accessed while this bug persists.”
→ This issue is caused here:
We only check for G and D, and in all other cases, call travel2service, which if the tripId is not Z, T or K, will result in the travel2service returning null.
Generally, null returned by called services is only rarely checked. It is unknown whether the developer did not implement null-checks by mistake, or whether the developer intentionally did not check for it, as the Exception will be caught by @Hystrix either way. Since the message of the fallback response is non-existent, this issue is assigned to SPECIAL CASE.
-
🔴 ASSURANCESERVICE:
”Doesn't allow users with the role of ADMIN, which is the highest in the system to add assurance to an order due to security misconfiguration.”
→ The securityConfig of the assurance service specifically only allows the role of user to access the endpoints, not the admin itself:
As we do not have a formal specification for this system, we cannot tell whether this was done intentionally or not. Depending on the security policy of the application, there could be cases where even administrators should have restricted access to specific endpoints for security or compliance reasons.
Integration Testing:
-
🟢 [F18a] POST /orderservice/order/query
”When querying for orders by travel date invalid results are returned as in the order-service implementation the travel-date is not compared to correct travel-start-date, but to incorrect bought-date-start instead.”
→ Instead of
The if statement should be:
Otherwise the implementation will not make sense logically.
-
🟢 [F18b] POST /api/v1/orderOtherService/orderOther/refresh
”When querying for orders by travel date invalid results are returned as in the order-service implementation the travel-date is not compared to correct travel-start-date, but to incorrect bought-date-start instead.”
→ Instead of
The if statement should be:
Otherwise the implementation will not make sense logically.
-
🔴 POST /api/v1/userservice/users/register
”When we send a request with an empty user's ID, we have a mismatch between the created user's IDs as the user service creates one itself as well as the ts-auth service. Thus this user cannot authenticate later on due to a mismatch between the user's IDs”
→ This does not seem to be the case. It is correct that when the userID is empty, the user service creates one itself:
The user service will then however call the auth service with the newly assigned userId:
There is also no such line of code as in the snipped above, where a new UUID is generated, in the ts-auth-service. Therefore, this is a NON-FAULT.
-
🟢 [F9] GET /api/v1/securityservice/securityConfigs/{accountId}
”Every request will result in a default @Hystrix response as there is an uncaught exception thrown when the service is trying to request security config from either ts-order-service or ts-order-other-service as the implementation uses date as part of the path of the request URL, which results in the wrong request and 400 HTTP Status Code is returned as conversion for this date is not defined in implementation.”
→ Endpoint of order service [F9a] and orderOther service [F9b] takes a Date object as a parameter in paths which will falsely be interpreted as a string. This will always lead to an IllegalArgumentException when attempting to call that endpoint
-
🔴 GET /api/v1/cancelservice/cancel/{orderId}/{loginId}
”The result of the drawback request o the inside-payment-service does not change the returned status.” on >>>
→ As the system lacks a formal specification, you could argue that some order just cannot be drawn back, but you still want the cancellation process to be successful (e.g., when the order is canceled too late or simply cannot be refunded)
-
🟢 [F6] GET /api/v1/foodservice/foods/{date}/{startStation}/{endStation}/{tripId} ,
”Travel service returns empty route even though the route service returns failure, therefore it causes exception on this endpoint.”
→ Root cause of this issue is within the travel service, NOT in food service. When route service in incapable of finding a route, it fails and will return null. The travel service however will configure an empty Route, which it cannot handle:
-
🔴 [F42] GET /api/v1/foodservice/foods/{date}/{startStation}/{endStation}/{tripId}
”Food map service does not do empty checks on an array, and it returns an empty list with success status which doesn't cause anything critical but a bug to fix.”
→ The food-map service correctly either returns a non-empty list, or null. Therefore, empty list checks are not required here.
-
🟣 [S1] POST api/v1/travelplanservice/travelPlan/transferResult
”No null check for the route of a trip response in TravelServiceImpl causes NullPointerException, which blocks the the whole process in case one of the trip's routes does not exist in the route repository.”
→ The NullPointerException will trigger a fallback response. The issue with the fallback response is that it remains unclear whether developers deliberately omitted preventative measures (e.g. null-check) in favor of relying on the fallback (which would be considered intended behavior), or not.
-
🟣 [S1] POST /api/v1/travelplanservice/travelPlan/cheapest
”No null check for the route of a trip response in TravelServiceImpl causes NullPointerException, which blocks the the whole process in case one of the trip's routes does not exist in the route repository.”
→ The NullPointerException will trigger a fallback response. The issue with the fallback response is that it remains unclear whether developers deliberately omitted preventative measures (e.g. null-check) in favor of relying on the fallback (which would be considered intended behavior), or not.
-
🟣 [S1] POST /api/v1/travelplanservice/travelPlan/quickest
”No null check for the route of a trip response in TravelServiceImpl causes NullPointerException, which blocks the the whole process in case one of the trip's routes does not exist in the route repository.”
→ The NullPointerException will trigger a fallback response. The issue with the fallback response is that it remains unclear whether developers deliberately omitted preventative measures (e.g. null-check) in favor of relying on the fallback (which would be considered intended behavior), or not.
Component Testing:
-
🔵 [D1] DELETE /api/v1/assuranceservice/assurances/orderid/{orderId}
”Implementation is misleading here: The response is dependent on whether we can still find the assurance after deleting it. However if the assurance never existed in the first place, the deletion doesn’t happen and we obviously can’t find the assurance in the database. Personally I think this should return a message, that the id wasn’t found in the database instead of a success, but their implementation is not generally wrong.”
→ repository.delete is performed before the check whether the object exists, always resulting in “Delete Successful”. This is not a mistake itself, because when no object exists, nothing happens when repository.delete is called. It is however odd to always respond with a successful delete, even when nothing was deleted.
-
🟢 [F13] POST /api/v1/basicservice/basic/travel
”The defined message “Train type doesn’t exist” is unreachable as the ts-price-service will always fail if the trainType is not found which leads to an error and a fallback response where everything is null.”
→ A request to an endpoint that does not exist is made (404)
-
🟢 [F10] GET /api/v1/cancelservice/cancel/refound/{orderId}
”The implementation of calculateRefund is incorrect as the calculated startTime uses the year. The documentation of Date() however explains that the year - 1990 should be provided as an argument for the year parameter. Therefore the nowDate will always be before the startTime and therefore always the price * 0.8 will be returned.”
→ Essentially, a Date object is initialized in a wrong way, making an if-branch unreachable
-
🔴 [F17] GET /api/v1/cancelservice/cancel/{orderId}/{loginId}
”The implementation seems to be wrong here as it doesn’t make sense to return a success if something fails.”
→ As there is no specification for the system, we cannot tell whether the drawback method is supposed to lead to a fail in the cancel service as well, as you could argue that some orders just cannot be drawn back (f.e. when you cancel the order too late), but you still want to successfully cancel this order
-
🔵 [D6] POST /api/v1/contactservice/contacts/admin
”TEST FAILED, method is implemented wrongly.”
→ In the ts-contact-service, a method verifies whether a contact ID exists before creating and saving a new contact object to the database. However, the controller method responsible for invoking this check generates a new random ID beforehand. As a result, the service consistently successfully saves a new contact object to the database, allowing the creation of multiple contact objects with the same information but different IDs. As the system lacks formal specification, we do not know whether the system is behaving as it is designed to. It could be argued that this specific implementation was intentional, because the system consistently creates contact objects with a unique ID in the database. Therefore, to grant the original developer the benefit of the doubt, this implementation was deemed as intentional. However, this issue is a poor design decision for two reasons: (a) the controller prematurely assigns IDs, which should be the responsibility of the service layer, and (b) the duplicate check on ID is redundant when a unique ID is generated beforehand.
-
🔵 [D1] DELETE /api/v1/contactservice/contacts/{contactsId}
”TEST FAILED, method is implemented wrongly.”
→ repository.delete is performed before the check whether the object exists, always resulting in “Delete Successful”. This is not a mistake itself, because when no object exists, nothing happens when repository.delete is called. It is however odd to always respond with a successful delete, even when nothing was deleted.
-
🟢 [F4a] POST /api/v1/foodmapservice/foodstores
”FAILED, method is implemented wrongly.”
→ null check instead of empty list check → result will always be successful, although it should not.
-
🔴 GET /api/v1/foodmapservice/trainfoods
”FAILED, method implementation is wrong.”
→ As tests do not fail locally, and the description itself is not very helpful, this mention was deemed as no fault.
-
🟢 [F4a] GET /api/v1/foodmapservice/trainfoods/{tripId}
”FAILED, method implementation is wrong.”
→ null check instead of empty list check → result will always be successful, although it should not.
-
🟢 [F4b] GET /api/v1/inside_pay_service/inside_payment/drawback/{userId}/{money}
”FAILED, method implementation is wrong.”
→ null check instead of empty list check → result will always be successful, although it should not.
-
🟢 [F4b] GET /api/v1/inside_pay_service/inside_payment/money
”FAILED, method implementation is wrong.”
→ null check instead of empty list check → result will always be successful, although it should not.
-
🟣 [S1] POST /api/v1/orderOtherService/orderOther
”This test case fails because the current implementation of the equals and hashCode methods in the Order class does not correctly identify duplicate orders based on the relevant fields that define an order's uniqueness. The equals method in the Order class should include fields such as accountId, trainNumber, from, to, seatClass, seatNumber, and travelDate to correctly identify if two orders are duplicates. The test case expects a status of 0 (indicating an order already exists), but due to the limitations in the equals method, it incorrectly allows the creation of a duplicate order and returns a status of 1 (success).”
→ The explanation seems to be incorrect. The implementation does correctly identify duplicate order based on all relevant fields. This is because the implementation uses .contains(order), which internally checks each value within the order and compares it with each element in the list of accountOrders.
The reason this test fails (and is presumably the reason why the author assumed that this service does not check for all relevant fields) is because the author of the test mistakenly did not fully configure the passed order for the requestBody. When at least one value of that Order object is null, a NullPointerException will be called when executing
.contains(order). This is because internally of that method, the service will try to executenull.equals(...).Since the NullPointerException will lead to the fallback Response, it is unknown whether the Developer knew that some error like that could happen here or not, because the message of the fallback response is empty. That is why this issue is assigned to SPECIAL CASE.
-
🟢 [F4f] GET /api/v1/orderOtherService/orderOther
”This test case fails because the current implementation of the getAllOrders method in the OrderOtherServiceImpl class incorrectly checks if the orders list is null instead of checking if the list is empty.”
-
🟣 [S1] POST /api/v1/orderservice/order
”This test case fails because the current implementation of the equals and hashCode methods in the Order class does not correctly identify duplicate orders based on the relevant fields that define an order's uniqueness. The equals method in the Order class should include fields such as accountId, trainNumber, from, to, seatClass, seatNumber, and travelDate to correctly identify if two orders are duplicates. The test case expects a status of 0 (indicating an order already exists), but due to the limitations in the equals method, it incorrectly allows the creation of a duplicate order and returns a status of 1 (success).”
→ The explanation seems to be incorrect. The implementation does correctly identify duplicate order based on all relevant fields. This is because the implementation uses .contains(order), which internally checks each value within the order and compares it with each element in the list of accountOrders.
The reason this test fails (and is presumably the reason why the author assumed that this service does not check for all relevant fields) is because the author of the test mistakenly did not fully configure the passed order for the requestBody. When at least one value of that Order object is null, a NullPointerException will be called when executing
.contains(order). This is because internally of that method, the service will try to executenull.equals(...).Since the NullPointerException will lead to the fallback Response, it is unknown whether the Developer knew that some error like that could happen here or not, because the message of the fallback response is empty. That is why this issue is assigned to SPECIAL CASE.
-
🔴 POST /api/v1/routeplanservice/routePlan/cheapestRoute
”This test case fails because the current implementation of the controller does not validate the input. The controller needs to be updated to use @Valid and handle validation errors to return a 400 status code.”
→ The explanation is incorrect because the tests are implemented incorrectly. The service calls travel service and travel2 service, which need to be mocked in the context of component tests. The mocking is however missing, which leads to a crash in form of a fallback response.
-
🔴 POST /api/v1/routeplanservice/routePlan/quickestRoute
”This test case fails because the current implementation of the controller does not validate the input. The controller needs to be updated to use @Valid and handle validation errors to return a 400 status code.”
→ The explanation is incorrect because the tests are implemented incorrectly. The service calls travel service and travel2 service, which need to be mocked in the context of component tests. The mocking is however missing, which leads to a crash in form of a fallback response.
-
🔴 POST /api/v1/routeplanservice/routePlan/minStopStations
”This test case fails because the current implementation of the controller does not validate the input. The controller needs to be updated to use @Valid and handle validation errors to return a 400 status code.”
→ The explanation is incorrect because the tests are implemented incorrectly. The service calls travel service and travel2 service, which need to be mocked in the context of component tests. The mocking is however missing, which leads to a crash in form of a fallback response.
-
🔵 [D1] DELETE /api/v1/routeservice/routes/{routeId}
”This test case fails because the method is not checking if the route exists before attempting to delete it. The method should return a status of 0 (indicating failure) when attempting to delete a non-existent route. To fix this, we should check if the route exists before attempting to delete it. If the route does not exist, return a failure response immediately”
→ repository.delete is performed before the check whether the object exists, always resulting in “Delete Successful”. This is not a mistake itself, because when no object exists, nothing happens when repository.delete is called. It is however odd to always respond with a successful delete, even when nothing was deleted.
-
🟢 [F5b] POST api/v1/stationservice/stations
this endpoint does not work as expected because the create method in service class is using
to determine if the station already exists. However, findById returns an Optional, meaning that if there is no element in the database that matches the ID, it will return Optional.empty instead of null. […]
-
🟢 [F5b] PUT api/v1/stationservice/stations
this endpoint does not work as expected because the create method in service class is using
to determine if the station already exists. However, findById returns an Optional, meaning that if there is no element in the database that matches the ID, it will return Optional.empty instead of null. […]
-
🟢 [F5b] DELETE api/v1/stationservice/stations
this endpoint does not work as expected because the create method in service class is using
to determine if the station already exists. However, findById returns an Optional, meaning that if there is no element in the database that matches the ID, it will return Optional.empty instead of null. […]
-
🔵 [D2] POST api/v1/stationservice/stations/idlist
”Whenever a name does not match any names of the elements in the database, an entry “Not Exist” is created and added to the list, which is returned in the end. Is it desired that the service returns a list containing “Not Exist” elements whenever a name could not be found in the database? In my opinion, names that do not match any elements in the database should not be included in the response.”
→ This is classified as an unconventional design choice, because if handled correctly, having “Not Exist” elements in the response is no mistake.
-
🟢 [F5a] POST api/v1/trainservice/trains
this endpoint does not work as expected because the create method in service class is using
to determine if the station already exists. However, findById returns an Optional, meaning that if there is no element in the database that matches the ID, it will return Optional.empty instead of null. […]
-
🟢 [F5a] PUT api/v1/trainservice/trains
this endpoint does not work as expected because the create method in service class is using
to determine if the station already exists. However, findById returns an Optional, meaning that if there is no element in the database that matches the ID, it will return Optional.empty instead of null. […]
-
🟢 [F5a] DELETE api/v1/trainservice/trains/{id}
this endpoint does not work as expected because the create method in service class is using
to determine if the station already exists. However, findById returns an Optional, meaning that if there is no element in the database that matches the ID, it will return Optional.empty instead of null. […]
-
🟢 [F4c] POST api/v1/travel2service/trips/routes
”[…] This means that in our case, if there are no trips that are associated with the given routeId, tempTripList will never be null, but rather an empty list […]”
-
🟢 [F1a] POST api/v1/travel2service/trips
”[…] the status code is set to 1 for both successful and unsuccessful responses. This inconsistency deviates from the expected behavior in some endpoints, could lead to confusion, and should be corrected.”
-
🟢 [F1a] PUT api/v1/travel2service/trips
”[…] the status code is set to 1 for both successful and unsuccessful responses. This inconsistency deviates from the expected behavior in some endpoints, could lead to confusion, and should be corrected.”
-
🟢 [F6] GET api/v1/travelservice/routes/{tripId}
”for when routeservice does not find a route with the routeId from the found trip, this endpoint does not work as expected. This is because of the way the method getRouteByRouteId is implemented in the service. If routeservice fails and returns null, getRouteByRouteId creates a new route with all values inside being null, and returns that instead […]”
→ when route service in incapable of finding a route, it fails and will return null. The travel service however will configure an empty Route, which which will lead to a check whether the list of returned routes is empty to always be false:
-
🟢 [F4d] POST api/v1/travelservice/trips/routes
”[…] This means that in our case, if there are no trips that are associated with the given routeId, tempTripList will never be null, but rather an empty list […]”
-
🟢 [F1b] POST api/v1/travelservice/trips
”[…] the status code is set to 1 for both successful and unsuccessful responses. This inconsistency deviates from the expected behavior in some endpoints, could lead to confusion, and should be corrected.”
-
🟢 [F1b] PUT api/v1/travelservice/trips
”[…] the status code is set to 1 for both successful and unsuccessful responses. This inconsistency deviates from the expected behavior in some endpoints, could lead to confusion, and should be corrected.”
-
🟢 [F19b] (F19a for travel2) POST api/v1/travelservice/trip_detail
”A request to the wrong endpoint is made → Something inside the request body is null and not checked, and that value will be needed in
api/v1/ticketinfoservice/ticketinfo/{name}but since name is null,api/v1/ticketinfoservice/ticketinfois called, which also exists” -
🔵 [D3] ts-user-service
”Only duplicate usernames are checked to not be in the database already. Generally, however, I would expect a service that persists users in a database to have unique ids for efficient database operations. This means that I would expect this service to return a response indicating that the user already exists (status 0: user already exists) instead. The following equivalence class has been written in a way as if this endpoint works as expected.”
→ The reason this is classified as an unconventional design choice is that this service is generally implemented to intentionally look for usernames and not for ids when searching for users in the database. Nonetheless, the whole point of an ID is to uniquely identify a user, and there is no reason for an ID to even exist if it is not unique.
-
🟢 [F11] POST api/v1/userservice/users/register
”There is a mismatch between the return type of the fallback method in UserController (returns HttpEntity) and the return type of this controller method registerUser (returns ResponseEntity), which causes all tests of this endpoint to immediately fail on start up. This issue happens here, because registerUser is annotated with @HystrixCommand in UserController. Either the return type needs to be adjusted, or the annotation with its fallback method needs to be removed in order to pass the tests.”
-
🟢 [F11] DELETE api/v1/userservice/users/{userId}
”There is a mismatch between the return type of the fallback method in UserController (returns HttpEntity) and the return type of this controller method registerUser (returns ResponseEntity), which causes all tests of this endpoint to immediately fail on start up. This issue happens here, because registerUser is annotated with @HystrixCommand in UserController. Either the return type needs to be adjusted, or the annotation with its fallback method needs to be removed in order to pass the tests.”
Integration Testing:
-
🔵 [D1] DELETE /api/v1/adminrouteservice/adminroute/{routeId}
”The implementation of the ts-route-service is probably wrong by mistake as in any case, no matter if a route exists with the routeId or not the response is success. This happens because after deleting the route based on the routeId from the repository, the service tries to find the route based on the routeId in the repository, which is obviously never the case.”
→ repository.delete is performed before the check whether the object exists, always resulting in “Delete Successful”. This is not a mistake itself, because when no object exists, nothing happens when repository.delete is called. It is however odd to always respond with a successful delete, even when nothing was deleted.
-
🔴 GET /api/v1/cancelservice/cancel/{orderId}/{loginId}
”In reality the service will still return a success when the drawback fails. In my opinion this is an unfavorable design decision of the ts-cancel-service as the drawback is an essential part of the cancellation process and if it fails the whole cancellation should fail. I presume that the behavior was not intended.”
→ We cannot tell for sure whether drawback is really an essential part of the cancellation process or not. You could argue that some orders can be canceled, but cannot be drawn back for specific reasons (f.e. when the order is canceled too late, or is simply not refundable in general).
-
🔴 GET /api/v1/cancelservice/cancel/{orderId}/{loginId}
”In my opinion this is an unfavorable design decision of the ts-cancel-service as the canceled order is already stored and the money is also already drawn back. So the main functionality of the endpoint (which presumably is canceling the order and money drawback) is a success but the endpoint still returns an error due to the ts-user-service.”
→ Since there is no formal specification for the system, we cannot tell whether it is intended to return an error when the ts-user-service fails. You could argue that having the user service to succeed is essential, as you need to associate the cancellation with the user to refund the money to that specific user.
-
🔴 GET /api/v1/foodservice/foods/{date}/{startStation}/{endStation}/{tripId}
”No integration tests are written for this equivalence class. The component test, where foodStoresListResult is null, fails because of the wrong method implementation.”
→ What wrong method implementation? As I see it, the food-map-service correctly either returns a non-empty list, or null, and the food service correctly checks for null values on the list
-
🔴 POST /api/v1/inside_pay_service/inside_payment
”unpossible to test, payment-service always gets valid data from the order-service, that is why why it always returns Success”
→ It is possible to test! (by creating an invalid order in the initData for the order service and force it to return that to the payment service).
-
🟢 [F14] POST /api/v1/inside_pay_service/inside_payment/difference
”because of the wrong method implementation logic, the payment-service would never been reached (wrong logic: if (totalExpand.compareTo(money) > 0) is always false). No tests are written.”
→ BigDecimal library is used incorrectly. This causes a service dependent on an if-condition to be never true. However, we do not see why this issue justifies not writing an integration test for.
-
🟣 [S1] POST /api/v1/orderOtherService/orderOther
”Failing Test Case: The service implementation does not validate null values, and therefore, does not return the expected error message.”
→ As the fallback response it triggered with a 200 OK upon encountering a NullPointerException, it remains unclear whether developers deliberately omitted preventative measures (e.g. null-checks) in favor of relying on the fallback (which would be considered intended behavior), or not.
-
🟣 [S1] POST /api/v1/orderOtherService/orderOther/refresh
”Failing Test Case: The service implementation does not check for non-existent stations and returns "Query Orders For Refresh Success".”
→ As the fallback response it triggered with a 200 OK upon encountering a NullPointerException, it remains unclear whether developers deliberately omitted preventative measures (e.g. null-checks) in favor of relying on the fallback (which would be considered intended behavior), or not.
-
🟣 [S1] GET /api/v1/orderOtherService/orderOther/orderPay/{orderId}
”The service implementation does not validate null values in the request body, leading to incorrect handling of orders with missing mandatory fields.”
→ As the fallback response it triggered with a 200 OK upon encountering a NullPointerException, it remains unclear whether developers deliberately omitted preventative measures (e.g. null-checks) in favor of relying on the fallback (which would be considered intended behavior), or not.
-
🟣 [S1] POST /api/v1/orderservice/order
”Failing Test Case: The service implementation does not validate null values, and therefore, does not return the expected error message”
→ As the fallback response it triggered with a 200 OK upon encountering a NullPointerException, it remains unclear whether developers deliberately omitted preventative measures (e.g. null-checks) in favor of relying on the fallback (which would be considered intended behavior), or not.
-
🟣 [S1] POST /api/v1/orderservice/order/refresh
”Failing Test Case: The service implementation does not check for non-existent stations and returns "Query Orders For Refresh Success".”
→ As the fallback response it triggered with a 200 OK upon encountering a NullPointerException, it remains unclear whether developers deliberately omitted preventative measures (e.g. null-checks) in favor of relying on the fallback (which would be considered intended behavior), or not.
-
🟣 [S1] GET /api/v1/orderservice/order/orderPay/{orderId}
”The service implementation does not validate null values in the request body, leading to incorrect handling of orders with missing mandatory fields.”
→ As the fallback response it triggered with a 200 OK upon encountering a NullPointerException, it remains unclear whether developers deliberately omitted preventative measures (e.g. null-checks) in favor of relying on the fallback (which would be considered intended behavior), or not.
-
🟢 [F9] (either order [F9a] or orderOther [F9b] service) POST /api/v1/preserveotherservice/preserveOther
”During the component and integration testing we discovered, that the ts-order-service and the ts-order-other-service both have big problems when getting a request to one of their endpoints that tries to receive a date in the path”
→ Root cause of this issue is within the order services, NOT in preserveother service. Endpoint takes a Date object as a parameter in paths which will falsely be interpreted as a string. This will always lead to an IllegalArgumentException when attempting to call that endpoint
-
🟢 [F9] (either order [F9a] or orderOther [F9b] service) POST /api/v1/preserveservice/preserve
”During the component and integration testing we discovered, that the ts-order-service and the ts-order-other-service both have big problems when getting a request to one of their endpoints that tries to receive a date in the path”
→ Root cause of this issue is within the order services, NOT in preserve service. Endpoint takes a Date object as a parameter in paths which will falsely be interpreted as a string. This will always lead to an IllegalArgumentException when attempting to call that endpoint
-
🟢 [F4e] POST api/v1/rebookservice/rebook
”[…] But since findByUserId returns a list, it can never be null. Instead, it returns an empty list if no Money instances are found. […]”
→ null check instead of empty list check, causes the drawBack method to always return a successful response.
-
🟢 [F23] POST api/v1/rebookservice/rebook
The Implementation of the service generally contains some return statements that cannot be reached. For example, the service checks the status of the order:
Although beforehand, it returns status code 0 with msg “you order not suitable to rebook” if the status is not equal to PAID. Therefore, none of these responses can be returned.
-
🔴 POST /api/v1/routeplanservice/routePlan/minStopStations
”Failing Test Case: The method implementation is incorrect. It returns "Success" even if the seat service is unavailable.”
→ The test case related to this description fails because “Success” is expected, but null is returned. Null is returned because when the seat service is unavailable, attempting to call this service results in a raised exception, which triggers the fallback response. Judging the nature of the fallback response, this is either intended or not. For this particular quote it seems that there has been a confusion with how the service behaves when the fallback response is triggered. Therefore, this mention is deemed as a NON-FAULT
-
🔴 POST /api/v1/routeplanservice/routePlan/cheapestRoute
”Failing Test Case: The method implementation is incorrect. It returns "Success" even if the seat service is unavailable.”
→ The test case related to this description fails because “Success” is expected, but null is returned. Null is returned because when the seat service is unavailable, attempting to call this service results in a raised exception, which triggers the fallback response. Judging the nature of the fallback response, this is either intended or not. For this particular quote it seems that there has been a confusion with how the service behaves when the fallback response is triggered. Therefore, this mention is deemed as a NON-FAULT
-
🟢 [F9a] POST /api/v1/preserveservice/preserve
”because these endpoints take a Date object as a parameter in paths which will falsely be interpreted as a string. This will always lead to an IllegalArgumentException in the orderservice / orderOtherService.”
→ Root cause of this issue is within the order services, NOT in preserve service. Endpoint takes a Date object as a parameter in paths which will falsely be interpreted as a string. This will always lead to an IllegalArgumentException when attempting to call that endpoint
-
🟢 [F9b] POST /api/v1/preserveotherservice/preserveOther
”because these endpoints take a Date object as a parameter in paths which will falsely be interpreted as a string. This will always lead to an IllegalArgumentException in the orderservice / orderOtherService.”
→ Root cause of this issue is within the order services, NOT in preserve service. Endpoint takes a Date object as a parameter in paths which will falsely be interpreted as a string. This will always lead to an IllegalArgumentException when attempting to call that endpoint
-
🟢 [F6] GET api/v1/travelservice/routes/{tripId}
”for when routeservice does not find a route with the routeId from the found trip, this endpoint does not work as expected. This is because of the way the method getRouteByRouteId is implemented in the service. If routeservice fails and returns null, getRouteByRouteId creates a new route with all values inside being null, and returns that instead […]”
→ when route service in incapable of finding a route, it fails and will return null. The travel service however will configure an empty Route, which which will lead to a check whether the list of returned routes is empty to always be false:
-
🔴 ts-user-service calling other services
”The way this service communicates with other services, in this case the ts-auth-service, is unique in comparison to most other services. This is because the userservice simply calls the auth service without making use of the response in any way. Therefore, the only way the authservice can influence the behavior of the userservice is when the authservice crashes or is unavailable.”
→ Treating an endpoint as void (even if called endpoint returns some data) is a NON-FAULT. It was likely also done on purpose here.
-
🟢 [F16] ts-preserve-service [F16a] / preserve-other-service [F16b]
”During testing I discovered that the timeout value of 5000 milliseconds causes a valid request to the endpoint to fail as a whole runthrough takes around 10-15 seconds. Therefore I adapted the timeout value of the endpoint of the ts-preserve-service to 20000 milliseconds, leaving some buffer for worse performing hardware devices.”
-
🟢 [F17] ts-order-service [F17a] / ts-order-other-service [F17b]
”The query method findByTravelDateAndTrainNumber in the getSoldTickets method of the orderservice and orderOtherService is likely not implemented correctly, because when there is an exact match for the travelDate and trainNumber, an unexpected error occurs, resulting in an InternalServerError and triggering a fallback response where everything is null”
→ findByTravelDateAndTrainNumber is implemented correctly. It is the NumberFormatException that is raised by executing
Integer.parseInt(tempOrder.getSeatNumber())that will lead to an InternalServerError. This is because the initData of the orderservices all have a SeatNumber of"FirstClass-30"which cannot be parsed to an Integer. -
🟢 [F3] ts-admin-travel-service
”The return of the endpoint is misleading: in this example we receive a list of admin trips however as the service implementation overrides the result object with the response of the first and the second request the final response contains the status message of the last request. This was probably done by mistake but might lead to confusion. Correct would be status 1 and success message but with this implementation it is 0 and no content as the response of ts-travel2-service is 0 and no content.”
→ getAllTravels in the ts-admin-travel-service calls both travel and travel2 to retrieve travel objects, and finally combines both responses to return a list of travel objects. If either travel or travel2 (or both) services return travel objects, the service should return a successful result containing a non-empty list of travel objects. However, the last service call overrides the result of the first response, meaning that the service will return a failure response, even when it did in fact find some objects.