fix: qualify agents_preprocess model refs with analysis dataset (#141)#148
Merged
Conversation
reddit_daily_summary and query_data_for_findings queried dbt models (agent_reddit_posts_daily, agent_reddit_sentiment_trends, etc.) by bare table name. BigQueryWarehouseResource.execute_query resolves unqualified names against its default economics_raw dataset, but agents_preprocess models materialize into economics_analysis, so every run failed with 404 Not found. Qualify these references with default_dataset_for_schema( "economics_analysis") — the same environment-suffix helper used for the staging-dataset fix in #143 — and add caller tests asserting the qualification. Closes #141 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
reddit_daily_summary_jobfailed on every run with404 Not found: ... economics_raw_dev.agent_reddit_posts_daily. The query referenced theagent_reddit_posts_dailydbt model by bare table name, butBigQueryWarehouseResource.execute_query()resolves unqualified names against its defaulteconomics_rawdataset — whileagents_preprocessmodels materialize intoeconomics_analysis.This PR qualifies those references using
default_dataset_for_schema("economics_analysis"), the same environment-suffix helper introduced for the staging-dataset fix in #143.Changes
news/news_summary_assets.py— the reddit_daily_summary_job: queries agent_reddit_posts_daily in wrong dataset (economics_raw instead of economics_analysis) #141 target: qualifyagent_reddit_posts_daily.data_points/data_point_finder.py— same defect, explicitly called out in reddit_daily_summary_job: queries agent_reddit_posts_daily in wrong dataset (economics_raw instead of economics_analysis) #141. Qualify all fiveagent_*references (agent_fred_series_latest_aggregates,agent_market_performance,agent_commodity_performance,agent_leading_econ_return_indicator,agent_reddit_sentiment_trends).test_bigquery_parameterized_callers.pyverifying both callers dataset-qualify theiragents_preprocessreferences.Scope note
A repo-wide sweep found the same defect class in ~60 more bare
agent_*references acrosseconomy_state/(data_access.py,domain_data_fetchers.py,economy_state_charts.py),investments/investment_recommendation_charts.py, andnarratives/economic_narrative_assets.py. Those are not in this PR — kept tight to #141's named scope. Filed as a follow-up issue.Testing
pytest tests/test_bigquery_parameterized_callers.py— 6 passed (4 existing + 2 new)pytest tests/test_data_point_finder.py— 25 passedCloses #141
🤖 Generated with Claude Code