[DEV-13391] Populate the rpt.transaction_download table in Delta#4583
Merged
sethstoudenmier merged 19 commits intoqatfrom Mar 11, 2026
Merged
[DEV-13391] Populate the rpt.transaction_download table in Delta#4583sethstoudenmier merged 19 commits intoqatfrom
sethstoudenmier merged 19 commits intoqatfrom
Conversation
…pending-api into ftr/dev-13391-populate-transaction-download-table
zachflanders-frb
previously approved these changes
Jan 29, 2026
| self.treasury_appropriation_account = spark.table("global_temp.treasury_appropriation_account") | ||
|
|
||
| @cached_property | ||
| def defc_by_group(self): |
Contributor
There was a problem hiding this comment.
Suggested change
| def defc_by_group(self): | |
| def defc_by_group(self) -> dict[str, list[str]]: |
…pending-api into ftr/dev-13391-populate-transaction-download-table
zachflanders-frb
approved these changes
Mar 10, 2026
sethstoudenmier
deleted the
ftr/dev-13391-populate-transaction-download-table
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description:
Populate the
rpt.transaction_downloadtable in Delta to match the contents of the Transaction Download from Monthly Downloads.Technical Details:
A new Spark DataFrame was created to manage the logic for the
rpt.transaction_downloadtable. Originally the intention was to use the silver tables (e.g.,int.transaction_normalized), however, during development I realized that there were some columns that the downloads use and are currently created onrpt.transaction_search. To avoid an even larger changeset than what I currently had the decision was made to userpt.transaction_searchinstead and create follow-up work that migrates to the use of silver tables.Additionally, the
rpt.transaction_downloadtable was originally created with column names that didn't match exactly the downloads orrpt.transaction_search. The intention was to use more normalized column names since our current transaction column names are sometimes inconsistent. After working to populate the table I made the decision to move away from this approach and instead use therpt.transaction_searchnames, making it easier to map 1:1. This also avoids the column names in the data from being tied too closely to the download column names since those could change.Requirements for PR Merge:
Explain N/A in above checklist: