Feature/ultip 5215#233
Open
grahammcvea-defra wants to merge 17 commits into
Open
Conversation
Introduced functionality to handle "Common Lands" data, including: - Added `SamCommonLandsEnabled` flag in `appsettings.json`. - Updated `SamDailyScanContext` and `SamHoldingImportContext` to include "Common Lands". - Enhanced `SamHoldingImportAggregationStep` to fetch and process "Common Lands". - Added mapping logic for "Common Lands" in Silver and Gold models. - Introduced new API routes and queries for "Common Lands". - Extended `IDataBridgeClient` to support "Common Lands" API calls. - Updated `SiteDto`, `SamHoldingDocument`, and `SiteDocument` to include "Common Lands" fields. - Added anonymization support for "Common Lands" in `PiiAnonymizerHelper`. - Enhanced `FakeDataBridgeClient` to simulate "Common Lands" data. - Added `SamCommonLandDailyScanStep` for daily scanning of "Common Lands". - Introduced `SamCommonLandMapper` for mapping "Common Lands" data. - Added new classes for "Common Lands" relationships and identifiers. These changes enable the application to retrieve, process, and map "Common Lands" data while ensuring proper anonymization and testing support.
Introduced nullability support in `ToAssociatedCommonLands` to handle null or empty lists. Added `PermanentLandHoldingIdentifier` to `SiteDocumentExtensions` for enhanced data mapping. Updated `SamHoldingImportOrchestratorTests` and `SamBulkImportWithAccurateRawDataTests` to include `commonLands` in test setups, mocking, and verification. Modified `GetAllQueryUris` to support `commonLandsUri`. Added mock setup for `GetByCodeAsync` to return "Common Land" site type. Enhanced `SamTestScenarios` with `RawCommonLandsByCommonCph` for test data. Added `SamCommonLandMapperTests` to validate mapping logic and `SamCommonLandTests` to ensure proper behavior of `SamCommonLand`. These changes ensure robust handling, mapping, and testing of `SamCommonLand` data.
Updated test cases in `SamCommonLandMapperTests.cs` and `SamCommonLandTests.cs` to replace `null` values for the `MAIN_CPH` property with `string.Empty`. This aligns with the updated behavior where `MAIN_CPH` no longer supports `null` values. Removed a test case in `SamCommonLandTests.cs` that validated `null` `MAIN_CPH`, as it is no longer relevant. Other test cases remain to validate empty and whitespace values for `MAIN_CPH`.
Updated the `NormaliseDate` method to use `CultureInfo.InvariantCulture` for both `DateTime.TryParse` and `ToString` to ensure consistent, culture-independent behavior. Added `System.Globalization` namespace to support these changes.
Enhanced `DataBridgeClientTests` with new unit tests to validate the behavior of `DataBridgeClient` methods for fetching "Sam Common Lands" data. Added tests for both enabled and disabled feature flag scenarios, covering paged responses and filtering by `COMMON_CPH`. Introduced helper methods in `MockSamData` to generate mock responses for "Sam Common Lands" data, ensuring realistic and consistent test data. These changes improve test coverage and verify functionality under various configurations.
Added a new test class `PiiAnonymizerHelperTests` to validate the `PiiAnonymizerHelper` class. Introduced tests to ensure PII fields in `SamCommonLand` are anonymized correctly, including `ADDRESS_LINE_1`, `POSTCODE`, `PREMISES_NAME`, `EASTING`, and `NORTHING`. Verified deterministic behavior based on `COMMON_CPH` and ensured non-PII fields remain unchanged. Added tests for edge cases such as null or empty fields, placeholder values, and empty `DataBridgeResponse` objects. Included a comprehensive test to validate anonymization of all PII fields in a single call.
Updated `VerifySilverDataTypesAsync` in both `SamImportHoldingAnonMessageTests` and `SamImportHoldingMessageTests` to add a filter excluding `SiteTypeCode` equal to "CL". Replaced `_localStackFixture` with `localStackFixture` in `SamImportHoldingMessageTests` for `CreateMessage` and `SendMessageAsync` calls to ensure consistent object usage. Updated `DisposeAsync` in `SamImportHoldingMessageTests` to use `mongoDbFixture` instead of `_mongoDbFixture` for purging data tables. These changes improve query precision, naming consistency, and maintainability.
Refactored `SamImportHoldingAnonMessageTests` and `SamImportHoldingMessageTests` to use private readonly fields for dependencies, improving encapsulation and maintainability. Updated method calls to align with the new structure. Added `SamCommonLandDailyScanStepTests` to test the functionality of `SamCommonLandDailyScanStep`. This includes mocking dependencies, configuring in-memory feature flags, and writing unit tests for various scenarios such as handling feature flags, API responses, pagination, and error handling. Introduced `FluentAssertions` for expressive assertions and improved test readability. Enhanced code consistency and maintainability through better naming conventions and encapsulation.
Added multiple test methods to validate anonymization logic: - `GetSamCommonLandsAsync_Generic_ShouldAnonymizeAddressAndLocationFields`: Ensures PII fields are anonymized while non-PII fields remain unchanged. - `GetSamCommonLandsByCommonCphAsync_ShouldAnonymizeAllCommonLands`: Verifies anonymization of all common lands for a given `COMMON_CPH`. - `GetSamCommonLandsByCommonCphAsync_ShouldHandleNullFields`: Confirms proper handling of null values in PII fields. - `GetSamCommonLandsByCommonCphAsync_ShouldNotAnonymizePlaceholderPremisesName`: Ensures placeholder values like `"-"` are not anonymized. - `GetSamCommonLandsAsync_ShouldProduceDeterministicResults`: Validates deterministic anonymization for the same `COMMON_CPH`. - `GetSamCommonLandsAsync_ShouldPreserveNonPiiFields`: Confirms non-PII fields are preserved while PII fields are anonymized. - `GetSamCommonLandsAsync_ShouldReturnNull_WhenInnerReturnsNull`: Ensures `null` is returned when the inner client returns `null`. - `GetSamCommonLandsByCommonCphAsync_ShouldHandleEmptyList`: Verifies proper handling of empty list responses.
Simplifies common land record identification by replacing IsDefinitionRecord/IsRelationshipRecord with IsMainCphPopulated. Updates ToSilver mapping to use only populated MAIN_CPH records for associations. Gold mapping step now enriches main sites asynchronously and tracks them for bulk update in persistence. Adds and updates unit tests for new logic, removes obsolete tests, and improves logging for main site updates.
Refactored logic to always update or add the main site in context.AssociatedMainSites, even if no new common land is added. Replaced shorthand list initialization for AssociatedCommonLands with explicit instantiation for clarity and compatibility.
Refactor tests to set AssociatedCommonLands as an empty list instead of null, aligning with non-nullable property changes. Replace count assertions with Assert.Single for improved clarity in SamHoldingImportPersistenceStepTests.cs.
Adjusted the first test to expect two captured items instead of one. Simplified the assertion in the second test to use Assert.Single for clarity and conciseness.
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ensure main site is always tracked in context list
Refactored logic to always update or add the main site in context.AssociatedMainSites, even if no new common land is added. Replaced shorthand list initialization for AssociatedCommonLands with explicit instantiation for clarity and compatibility.