Skip to content

BFD-4291: Update overshare_mbi logic - #2814

Merged
aschey-forpeople merged 11 commits into
masterfrom
BFD-4291
Sep 30, 2025
Merged

BFD-4291: Update overshare_mbi logic#2814
aschey-forpeople merged 11 commits into
masterfrom
BFD-4291

Conversation

@aschey-forpeople

@aschey-forpeople aschey-forpeople commented Sep 22, 2025

Copy link
Copy Markdown
Contributor

JIRA Ticket:
BFD-4291

What Does This PR Do?

  • Updates overshare logic to handle overshared MBIs based on recent findings related to IDR's merged process. There's a comment in the model explaining the details, but I'll paste it here for convenience:
The xref data in the bene_hstry table is not completely reliable
because sometimes HICNs can be reused, causing two records to be
xref'd even if they're not the same person.
We'll only trust xref records that have a valid entry in the bene_xref
table (this means it's coming from CME). For any MBIs tied to more than
one bene_sk that doesn't have a valid xref, we will prevent it from being
shown since it may be incorrectly linked to more than one person.
  • Replaces the overshare_mbis materialized view with a separate table that will be loaded from IDR each time. Materialized views are not smart enough to perform incremental refreshes, so this approach is actually substantially faster.

  • Refactors some of the larger Python functions in the pipeline to make the code a bit more readable.

What Should Reviewers Watch For?

If you're reviewing this PR, please check for these things in particular:

What Security Implications Does This PR Have?

Please indicate if this PR does any of the following:

  • Adds any new software dependencies

  • Modifies any security controls

  • Adds new transmission or storage of data

  • Any other changes that could possibly affect security?

  • I have considered the above security implications as it relates to this PR. (If one or more of the above apply, it cannot be merged without the ISSO or team security engineer's (@sb-benohe) approval.)

  • I have created tests to sufficiently ensure the reliability of my code, if applicable. If this is a modification to an existing piece of code, I have audited the associated tests to ensure everything works as expected.

Validation

Have you fully verified and tested these changes? Is the acceptance criteria met? Please provide reproducible testing instructions, code snippets, or screenshots as applicable.

Comment thread apps/bfd-pipeline/bfd-pipeline-idr/loader.py
Comment thread apps/bfd-pipeline/bfd-pipeline-idr/loader.py
@aschey-forpeople
aschey-forpeople marked this pull request as ready for review September 23, 2025 17:38
Comment thread apps/bfd-pipeline/bfd-pipeline-idr/bfd.sql Outdated
mel1-G
mel1-G previously approved these changes Sep 24, 2025
Comment thread apps/bfd-pipeline/bfd-pipeline-idr/bfd.sql Outdated

@np-complete0 np-complete0 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return type should be int here:
def transform_null_int(value: int | None) -> float:
in model.py

MahiFentaye
MahiFentaye previously approved these changes Sep 26, 2025
np-complete0
np-complete0 previously approved these changes Sep 26, 2025
@bfd-sast

bfd-sast Bot commented Sep 27, 2025

Copy link
Copy Markdown

Quality Gate passed Quality Gate passed for 'bfd-parent'

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

@aschey-forpeople
aschey-forpeople added this pull request to the merge queue Sep 30, 2025
Merged via the queue into master with commit 2d1bc47 Sep 30, 2025
13 checks passed
@aschey-forpeople
aschey-forpeople deleted the BFD-4291 branch September 30, 2025 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants