Simplify the pinval.vw_assessment_card data model#856
Conversation
… always match predictors
…-modelassessment_card-in-pinvalvw_assessment_card
…training data table
| ## meta_card_num | ||
|
|
||
| {% docs meta_card_num %} | ||
| {% docs column_pinval_meta_card_num %} |
There was a problem hiding this comment.
Not sure how I didn't catch this in #851, but this docs block identifier should follow our standard {type}_{schema}_* naming pattern.
|
|
||
| {% enddocs %} | ||
|
|
||
| ## model_run_id |
There was a problem hiding this comment.
These docs blocks are not being deleted, but have rather been renamed and moved to other parts of the file in order to follow our block sorting rules.
| -- Join on year rather than run ID because `model.training_data` is | ||
| -- guaranteed to be unique by year but may have a different run ID | ||
| -- than the comps run |
There was a problem hiding this comment.
This is a little bit confusing, and notably it means that pinval.vw_comp will have a run ID that matches pinval.vw_assessment_card for now (2025-02-11-charming-eric), since model.training_data and pinval.vw_assessment_card both use the final 2025 model. There's probably a better way to solve this, but one of my tasks for the month is to figure out a permanent solution to the final/comp model run discrepancy issue, so I think an imperfect solution is fine for now.
wagnerlmichael
left a comment
wagnerlmichael
left a comment
There was a problem hiding this comment.
Looks good to me, just a question to confirm my understanding of one of the data tests.
| -- included in the assessment set for a given model run. We need these | ||
| -- essential attrs even when parcels aren't in the assessment set in order | ||
| -- to generate detailed descriptions for why those parcels don't have | ||
| -- reports |
There was a problem hiding this comment.
[Praise]: Very clear docs
| AND table_name = '{{ ref("pinval.vw_assessment_card").identifier }}' | ||
| ) | ||
| ) | ||
| ) = 0 |
There was a problem hiding this comment.
[Question]: So the ARRAY_EXCEPT acts as a set difference, and then CARDINALITY checks to see if there is anything returned from that set difference?
There was a problem hiding this comment.
Yup, you've got it! I clarified the comment in 5952ee7 to make this explicit.
jeancochrane
deleted the
jeancochrane/855-avoid-selecting-star-from-modelassessment_card-in-pinvalvw_assessment_card
branch
…#85) This PR updates our Hugo report generation code to handle the changes to the `pinval.vw_assessment_card` data model that we're making in ccao-data/data-architecture#856. In addition, we fold in one small change to improve the experience of testing changes to the data model: Factoring out constants for `PINVAL_ASSESSMENT_CARD_TABLE` and `PINVAL_COMP_TABLE`. By centralizing the references to these two tables, we can more easily change them when we need to test a corresponding change in `data-architecture`. See PIN `06171140020000` on the staging site for an example of a deployed report. Connects ccao-data/data-architecture#855.
2 participants
This PR follows up on the changes we made to
pinval.vw_assessment_cardin #851 to add rows for every parcel in the County for every model run. That PR used a confusing convention ofparcel_*andmodel_*column name prefixes for new columns in order to skirt around the problem that we select*frommodel.assessment_cardand so can't rename any of its columns. This PR resolves this problem at the root by instead selecting the specific columns that we want frommodel.assessment_cardsuch that we can filter them, rename them, or transform them.Note that this choice introduces a maintenance burden, in that we have to manually add any new predictors to the set of selected columns in
pinval.vw_assessment_cardwhenever we add new predictors to the res model. We make two changes to mitigate the risk that we will forget to do this:model_predictor_all_namearray in this table is a subset of the columns of thepinval.vw_assessment_cardtablemodel-res-avmto update the annual model finalization checklist to include an item for updatingpinval.vw_assessment_cardto include any new predictors (Add item to annual model issue checklist for double-checkingpinvalresources model-res-avm#384)This PR also folds in one small unrelated data model change to
pinval.vw_comp, switching from the temporarymodel.pinval_test_training_datatable to the permanentmodel.training_datatable that we added in #804 in order to augment comps with training data characteristics. That change resolves #803.Connects #855.
Companion PRs:
pinval.vw_assessment_carddata model homeval#85pinvalresources model-res-avm#384