#146: Implement application_analytics_request Lambda with real DB queries#167
Open
dokababa wants to merge 3 commits into
Open
#146: Implement application_analytics_request Lambda with real DB queries#167dokababa wants to merge 3 commits into
dokababa wants to merge 3 commits into
Conversation
…th real DB queries - Adds request volume trend queries (7-day/30-day daily, 1-year monthly) - Adds category x country aggregation with filter_category, filter_country, sort_by - Adds top 5 countries with rank - Wraps DB connection in try/except (500 on failure) - Wraps query functions in try/except (returns [] on error) - Closes cursor and connection in finally block - Reads DB credentials from os.environ (no hardcoded secrets) - Adds mock_local_db.py SQLite harness for local testing without DB access - Adds test_application_analytics_request.py with 13 unit tests covering all Phase 2 verifications
dokababa
force-pushed
the
dokababa_146_application_analytics_api
branch
from
22b4ea7 to
683df2a
Compare
Author
|
Code reviewed and deployed to AWS Lambda as requestApplicationAnalytics. Test event with {} returned statusCode 200 with all 5 keys populated from the Virginia DB. |
… feedback - Replace plain env-var DB credentials with AWS Systems Manager Parameter Store via boto3 (reads /dev/saayam/db/Virginia/Analytics/user) to match the convention used by the other analytics Lambdas (beneficiariesTrendAnalysis, volunteerApplicationAnalytics) - Add CORS headers (Access-Control-Allow-Origin/Headers/Methods) so the response can be consumed by the webapp - Switch country lookups from country_name to country_code in top_countries and requests_by_category_region (returns 3-letter ISO codes: AFG, USA, ALA, etc.) - Use UPPER(c.country_code) = %s for case-insensitive country matching - Rename event filter keys from filter_category/filter_country to category/country per UI team request - Update mock tests to stub boto3 and patch get_db_config so they run without AWS access Verified end-to-end on deployed Lambda (requestApplicationAnalytics) against the live Virginia RDS: - Status 200 baseline with all 5 keys populated - category-only filter returns matching category rows - country-only filter returns matching country rows - both-filter combination returns single matching row
…gation Per reviewer feedback, move the request-volume aggregation out of pandas and into SQL so all grouping happens in PostgreSQL. Changes: - get_request_volume_trend now uses DATE_TRUNC(unit, submission_date) with GROUP BY and ORDER BY in a single query, instead of pulling raw submission_date rows and using pandas to_period / groupby - Drop 'import pandas as pd' from the Lambda (no longer used anywhere) - Update test mocks: SAMPLE_DATES now returns 2-column (date, count) tuples to match the new SELECT shape - Update tests to json.loads(response["body"]) since the body is a JSON-encoded string; also rename filter event keys in tests to category/country Verified on deployed Lambda (requestApplicationAnalytics) against the live Virginia RDS — output format is identical to the pandas version, cold-start ~35% faster, memory drops from 125MB to 97MB.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements
application_analytics_request.pyas a real AWS Lambda function querying the Virginia PostgreSQL DB for the Super Admin dashboard analytics.Closes #146 (pending Phase 3 deployment).
Phase 1 — Code (complete):
filter_category,filter_country,sort_byparamsfinallyblockif __name__ == "__main__"block for local testingPhase 2 — Tested locally:
test_application_analytics_request.py(13 unit tests using mocks) covering: keys present, filters, empty DB, connection failures, finally-block closuremock_local_db.py— in-memory SQLite harness mirroring the real schema for local dev without DB credentialsPhase 4 — Security:
os.environonly — nothing hardcodedPending
Tables/fields used
request(req_user_id, req_cat_id, submission_date),users(user_id, country_id),country(country_id, country_name),help_categories(cat_id, cat_name)