AB#241235: add ! push data to HOPE core #45

vitali-yanushchyk-valor · 2025-03-07T14:19:40Z

Legal Boilerplate

Look, I get it.
Contributing to HOPE Workspace, I retain all rights, title and interest in and to my contributions, and by keeping
this boilerplate intact I confirm that HOPE Workspace can use, modify, copy, and redistribute my contributions,
under HOPE's choice of terms.

codecov · 2025-03-12T13:31:18Z

Codecov Report

Attention: Patch coverage is 92.69663% with 13 lines in your changes missing coverage. Please review.

Project coverage is 92.40%. Comparing base (d0f938f) to head (849a197).
Report is 6 commits behind head on develop.

Files with missing lines	Patch %	Lines
...try_workspace/workspaces/admin/cleaners/actions.py	47.05%	9 Missing ⚠️
src/country_workspace/contrib/hope/push.py	96.22%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop      #45      +/-   ##
===========================================
+ Coverage    91.18%   92.40%   +1.22%     
===========================================
  Files          151      152       +1     
  Lines         4320     4466     +146     
  Branches       369      389      +20     
===========================================
+ Hits          3939     4127     +188     
+ Misses         283      236      -47     
- Partials        98      103       +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

domdinicola · 2025-03-17T15:31:56Z

@vitali-yanushchyk-valor pls resolve conflicts

domdinicola · 2025-03-18T11:11:42Z

src/country_workspace/contrib/hope/push.py

+
+    def __post_init__(self) -> None:
+        """Initialize the base path for API requests."""
+        self.base_path = f"{self.co_slug}/rdi/"


why not using init?

it's dataclass

domdinicola · 2025-03-18T11:12:27Z

src/country_workspace/contrib/hope/push.py

+        """Initialize the base path for API requests."""
+        self.base_path = f"{self.co_slug}/rdi/"
+
+    def validate_households(self) -> None:


shouldn't we have this in another place and call it? looks to me duplicated code

This function only validates the selected beneficiaries, without updating last_checked and add a new version.

domdinicola · 2025-03-18T11:13:32Z

src/country_workspace/contrib/hope/push.py

+        """
+        try:
+            with transaction.atomic():
+                households = list(CountryHousehold.objects.filter(id__in=successful_ids))


why converting it in list?

domdinicola · 2025-03-18T11:52:50Z

src/country_workspace/contrib/hope/push.py

+
+    """
+    processor = PushProcessor(
+        queryset=CountryHousehold.objects.filter(pk__in=job.config["pks"]),


would we have problems with large datasets?

domdinicola · 2025-03-18T11:54:05Z

src/country_workspace/contrib/hope/constants.py

 # NEVER CHANGE THIS VALUES
-HOUSEHOLD_CHECKER_NAME = "HOPE Household core"
-INDIVIDUAL_CHECKER_NAME = "HOPE Individual core"
+HOUSEHOLD_CHECKER_NAME: Final[str] = "HOPE Household checker"


core checker & also for individaul

@vitali-yanushchyk-valor

domdinicola · 2025-03-19T09:33:23Z

@vitali-yanushchyk-valor fix tests pls

domdinicola · 2025-03-19T17:16:59Z

src/country_workspace/contrib/hope/constants.py

 # NEVER CHANGE THIS VALUES
-HOUSEHOLD_CHECKER_NAME = "HOPE Household core"
-INDIVIDUAL_CHECKER_NAME = "HOPE Individual core"
+HOUSEHOLD_CHECKER_NAME: Final[str] = "HOPE Household checker"


@vitali-yanushchyk-valor

domdinicola · 2025-03-20T13:13:13Z

src/country_workspace/versioning/checkers.py

-    hh_fs.fields.get_or_create(name="consent", attrs={"required": False, "label": "Consent"}, definition=_bool)
-    hh_fs.fields.get_or_create(name="consent_sharing", definition=_bool)
-    hh_fs.fields.get_or_create(name="country_origin", definition=_char)
-    hh_fs.fields.get_or_create(name="first_registration_date", attrs={"label": "First Registration"}, definition=_date)


why we removed this?

datachecker changes follow the current HOPE core.

consent_sharing - is not bool - it's ConsentSharingEnum and when I asked about it at the sync, I got an answer, that we will not use it.

country_origin - I declared to use CountryChoice

first_registration_date at the HOPE core is defined like serializers.DateTimeField(default=timezone.now)

I updated the system to require full_name, birth_date, and relationship too, since HOPE core will respond with an error if they are not provided.

This is one of the questions I wanted to clarify. We have a base fieldset, and in CW we can create additional fieldsets that extend existing. Since this is the very first commit regarding data push to HOPE core —and we don't yet have such a process in production—I modified the default datachecker in accordance with the current requirements from HOPE core. In the future, this mechanism for creating new fieldsets can be utilized.
Of course, if the base datachecker is meant to remain exactly as it was before my changes, I can revert them.

domdinicola · 2025-03-20T13:13:20Z

src/country_workspace/versioning/checkers.py

    _h_residence = FieldDefinition.objects.get(slug="hope-hh-residencestatus")
    _i_gender = FieldDefinition.objects.get(slug="hope-ind-gender")
    _i_disability = FieldDefinition.objects.get(slug="hope-ind-disability")
    _i_role = FieldDefinition.objects.get(slug="hope-ind-role")
    _i_relationship = FieldDefinition.objects.get(slug="hope-ind-relationship")

    hh_fs, __ = Fieldset.objects.get_or_create(name=HOUSEHOLD_CHECKER_NAME)
-    hh_fs.fields.get_or_create(name="address", attrs={"label": "Address", "required": True}, definition=_char)
+    hh_fs.fields.get_or_create(name="address", definition=_char)


why was this changed?

because currently address in not mandatory at the HOPE core side.

vitali-yanushchyk-valor force-pushed the feature/post2hope branch 2 times, most recently from 7b6430d to 2c2f794 Compare March 12, 2025 13:26

ntrncic requested a review from saxix March 12, 2025 14:53

vitali-yanushchyk-valor marked this pull request as ready for review March 12, 2025 14:54

vitali-yanushchyk-valor force-pushed the feature/post2hope branch 4 times, most recently from c2e94f7 to f614929 Compare March 17, 2025 14:29

vitali-yanushchyk-valor force-pushed the feature/post2hope branch from f614929 to 1f1aa31 Compare March 18, 2025 07:50

domdinicola reviewed Mar 18, 2025

View reviewed changes

vitali-yanushchyk-valor force-pushed the feature/post2hope branch 2 times, most recently from 51f2d42 to e38ae85 Compare March 18, 2025 14:36

vitali-yanushchyk-valor changed the title ~~add ! push data to HOPE core~~ AB#241235: add ! push data to HOPE core Mar 20, 2025

vitali-yanushchyk-valor added 4 commits March 20, 2025 08:10

add ! push data to HOPE core

3316dec

chg ! tests & rebase

c8f9206

chg ! refactor push to HOPE core

4028d29

chg ! rebase & refactor

edee7b1

vitali-yanushchyk-valor force-pushed the feature/post2hope branch from e38ae85 to edee7b1 Compare March 20, 2025 11:36

chg ! documentation

849a197

domdinicola reviewed Mar 20, 2025

View reviewed changes

domdinicola merged commit 22ef82c into develop Mar 20, 2025
28 checks passed

vitali-yanushchyk-valor deleted the feature/post2hope branch March 26, 2025 06:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AB#241235: add ! push data to HOPE core #45

AB#241235: add ! push data to HOPE core #45

vitali-yanushchyk-valor commented Mar 7, 2025

codecov bot commented Mar 12, 2025 •

edited

Loading

domdinicola commented Mar 17, 2025

domdinicola Mar 18, 2025

vitali-yanushchyk-valor Mar 18, 2025

domdinicola Mar 18, 2025

vitali-yanushchyk-valor Mar 20, 2025

domdinicola Mar 18, 2025

domdinicola Mar 18, 2025

vitali-yanushchyk-valor Mar 20, 2025

domdinicola Mar 18, 2025

domdinicola Mar 19, 2025

domdinicola commented Mar 19, 2025

domdinicola Mar 19, 2025

domdinicola Mar 20, 2025

vitali-yanushchyk-valor Mar 20, 2025

vitali-yanushchyk-valor Mar 20, 2025

domdinicola Mar 20, 2025

vitali-yanushchyk-valor Mar 20, 2025 •

edited

Loading

AB#241235: add ! push data to HOPE core #45

AB#241235: add ! push data to HOPE core #45

Conversation

vitali-yanushchyk-valor commented Mar 7, 2025

Legal Boilerplate

codecov bot commented Mar 12, 2025 • edited Loading

Codecov Report

domdinicola commented Mar 17, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

domdinicola commented Mar 19, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vitali-yanushchyk-valor Mar 20, 2025 • edited Loading

Choose a reason for hiding this comment

codecov bot commented Mar 12, 2025 •

edited

Loading

vitali-yanushchyk-valor Mar 20, 2025 •

edited

Loading