Skip to content

[FEATURE] Add redshift datasource api decorator. #11097

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: develop
Choose a base branch
from

Conversation

billdirks
Copy link
Contributor

  • Description of PR changes above includes a link to an existing GitHub issue
  • PR title is prefixed with one of: [BUGFIX], [FEATURE], [DOCS], [MAINTENANCE], [CONTRIB], [MINORBUMP]
  • Code is linted - run invoke lint (uses ruff format + ruff check)
  • Appropriate tests and docs have been updated

For more information about contributing, visit our community resources.

After you submit your PR, keep the page open and monitor the statuses of the various checks made by our continuous integration process at the bottom of the page. Please fix any issues that come up and reach out on Slack if you need help. Thanks for contributing!

Copy link

netlify bot commented Apr 14, 2025

Deploy Preview for niobium-lead-7998 ready!

Name Link
🔨 Latest commit
🔍 Latest deploy log https://app.netlify.com/sites/niobium-lead-7998/deploys/67fea9467be11037dc29ffa6
😎 Deploy Preview https://deploy-preview-11097.docs.greatexpectations.io
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link

codecov bot commented Apr 14, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.19%. Comparing base (7f0ee6f) to head (55cc36b).

✅ All tests successful. No failed tests found.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop   #11097      +/-   ##
===========================================
- Coverage    81.12%   79.19%   -1.93%     
===========================================
  Files          499      499              
  Lines        41330    41332       +2     
===========================================
- Hits         33528    32733     -795     
- Misses        7802     8599     +797     
Flag Coverage Δ
3.10 70.40% <100.00%> (+<0.01%) ⬆️
3.10 athena or openpyxl or pyarrow or project or sqlite or aws_creds ?
3.10 aws_deps ?
3.10 big ?
3.10 clickhouse ?
3.10 filesystem ?
3.10 mssql ?
3.10 mysql ?
3.10 postgresql ?
3.10 spark_connect ?
3.11 70.39% <100.00%> (-0.02%) ⬇️
3.11 aws_deps ?
3.11 big ?
3.11 clickhouse ?
3.11 filesystem ?
3.11 mssql ?
3.11 mysql ?
3.11 postgresql ?
3.11 spark_connect ?
3.12 70.40% <100.00%> (-0.02%) ⬇️
3.12 athena or openpyxl or pyarrow or project or sqlite or aws_creds 57.03% <100.00%> (+<0.01%) ⬆️
3.12 aws_deps ?
3.12 big ?
3.12 bigquery 49.49% <100.00%> (+<0.01%) ⬆️
3.12 databricks 51.12% <100.00%> (+<0.01%) ⬆️
3.12 filesystem ?
3.12 gx-redshift 48.69% <100.00%> (+<0.01%) ⬆️
3.12 mssql ?
3.12 mysql 52.28% <100.00%> (+<0.01%) ⬆️
3.12 postgresql 55.08% <100.00%> (+<0.01%) ⬆️
3.12 redshift 48.71% <100.00%> (+0.03%) ⬆️
3.12 snowflake 51.89% <100.00%> (+<0.01%) ⬆️
3.12 spark 58.20% <100.00%> (+<0.01%) ⬆️
3.12 spark_connect ?
3.12 trino ?
3.9 70.43% <100.00%> (+0.01%) ⬆️
3.9 athena or openpyxl or pyarrow or project or sqlite or aws_creds 57.03% <100.00%> (+<0.01%) ⬆️
3.9 aws_deps 46.79% <100.00%> (+<0.01%) ⬆️
3.9 big 55.14% <100.00%> (+<0.01%) ⬆️
3.9 bigquery 49.49% <100.00%> (+<0.01%) ⬆️
3.9 clickhouse ?
3.9 databricks ?
3.9 filesystem ?
3.9 gx-redshift 48.69% <100.00%> (+<0.01%) ⬆️
3.9 mssql ?
3.9 mysql ?
3.9 postgresql 55.06% <100.00%> (+<0.01%) ⬆️
3.9 redshift ?
3.9 snowflake 51.89% <100.00%> (+<0.01%) ⬆️
3.9 spark ?
3.9 spark_connect ?
3.9 trino ?
cloud ?
docs-basic 54.07% <100.00%> (+<0.01%) ⬆️
docs-creds-needed 52.93% <100.00%> (+<0.01%) ⬆️
docs-spark ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@billdirks billdirks changed the title Add redshift datasource api decorator. [MINORBUMP] Add redshift datasource api decorator. Apr 14, 2025
@billdirks billdirks marked this pull request as ready for review April 14, 2025 22:09
@billdirks billdirks changed the title [MINORBUMP] Add redshift datasource api decorator. [FEATURE] Add redshift datasource api decorator. Apr 15, 2025
Copy link
Contributor

@klavavej klavavej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some suggestions/questions about the copy

@@ -1,6 +1,6 @@
{
"title": "RedshiftDatasource",
"description": "Adds a redshift datasource to the data context using psycopg2.\n\nArgs:\n name: The name of this redshift datasource.\n connection_string: The SQLAlchemy connection string used to connect to the redshift\n database. This will use a redshift with psycopg2. For example:\n \"redshift+psycopg2://[email protected]:5439/database\"\n assets: An optional dictionary whose keys are TableAsset or QueryAsset names and whose\n values are TableAsset or QueryAsset objects.",
"description": "--Public API--Adds a redshift datasource to the data context using psycopg2.\n\nArgs:\n name: The name of this redshift datasource.\n connection_string: The SQLAlchemy connection string used to connect to the redshift\n database. This will use a redshift with psycopg2. For example:\n \"redshift+psycopg2://[email protected]:5439/database\"\n assets: An optional dictionary whose keys are TableAsset or QueryAsset names and whose\n values are TableAsset or QueryAsset objects.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is maybe the wrong place to apply these changes (looks like this is generated json?) but here's a suggestion that does a few things:

  • capitalizes Redshift as a proper noun
  • removes what appears to be some stray words (when compared to the similar page for postgres
Suggested change
"description": "--Public API--Adds a redshift datasource to the data context using psycopg2.\n\nArgs:\n name: The name of this redshift datasource.\n connection_string: The SQLAlchemy connection string used to connect to the redshift\n database. This will use a redshift with psycopg2. For example:\n \"redshift+psycopg2://[email protected]:5439/database\"\n assets: An optional dictionary whose keys are TableAsset or QueryAsset names and whose\n values are TableAsset or QueryAsset objects.",
"description": "--Public API--Adds a Redshift datasource to the data context using psycopg2.\n\nArgs:\n name: The name of this Redshift datasource.\n connection_string: The SQLAlchemy connection string used to connect to the Redshift\n database. For example:\n \"redshift+psycopg2://[email protected]:5439/database\"\n assets: An optional dictionary whose keys are TableAsset or QueryAsset names and whose\n values are TableAsset or QueryAsset objects.",

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to double check the connection string format here since it's different that what I'm using in my docs PR based on feedback on my docs plan

This PR has

redshift+psycopg2://[email protected]:5439/database

My PR has

redshift+psycopg2://<USER_NAME>:<PASSWORD>@<HOST>:<PORT>/<DATABASE>?sslmode=<SSLMODE>

notable differences are:

  • whether or not password is included
  • whether or not SSL mode is included

Should the discrepancy between PRs be resolved? if yes, in what direction?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants