Skip to content

V0.1.1 Adding pytest for bigquery #82

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 27 commits into
base: v0.1.1
Choose a base branch
from
Open

V0.1.1 Adding pytest for bigquery #82

wants to merge 27 commits into from

Conversation

tusharchou
Copy link
Owner

@tusharchou tusharchou commented Nov 4, 2024

Screenshot 2024-11-04 at 7 25 18 PM

Name: pytest for bigquery
About: Implement a test for class source bigquery
title: ''
labels: ''
assignees: ''


Is your pull request related to a issue? Please tag.

Describe the solution you have implemented
Describe the solution you have implemented

I implemented a test for the source_bigquery class using pytest. This includes covering various scenarios to ensure the class behaves as expected with different inputs and configurations.

Describe why alternatives you considered are not optimal

Using unittest was considered, but pytest was chosen for its concise and readable syntax, making the tests easier to maintain and understand.

Additional context

The tests were conducted against a mock BigQuery setup to ensure they can be executed in a local environment without needing access to an actual BigQuery instance.

Here's a screenshot of the test results:

tusharchou and others added 25 commits October 30, 2024 19:09
* Create publish.yml

* Update publish.yml
* Create publish.yml

* Update publish.yml

* Update publish.yml
* Update publish.yml

* Update publish.yml

* Update publish.yml
* Update publish.yml

* Update publish.yml

* Update publish.yml

* Update publish.yml

updated yaml file to copy distribution o/p from build to root directory

* Update publish.yml

added detailed copy from /ldf/dist to gihub/wo../dist

---------

Co-authored-by: Tushar Choudhary <[email protected]>
* Update publish.yml

* Update publish.yml

* Update publish.yml

* Update publish.yml

updated yaml file to copy distribution o/p from build to root directory

* Update publish.yml

added detailed copy from /ldf/dist to gihub/wo../dist

* Update publish.yml

added a code for creating dist directory

---------

Co-authored-by: Tushar Choudhary <[email protected]>
* Update publish.yml

* Update publish.yml

* Update publish.yml

* Update publish.yml

updated yaml file to copy distribution o/p from build to root directory

* Update publish.yml

added detailed copy from /ldf/dist to gihub/wo../dist

* Update publish.yml

added a code for creating dist directory

* Update publish.yml

relocating dots from ./github to /.github

---------

Co-authored-by: Tushar Choudhary <[email protected]>
* Update publish.yml

* Update pyproject.toml
* Update publish.yml

* Update pyproject.toml

* Update publish.yml
* Release v1.1 dist changes

* Release v1.1 publish.yml changes

* Release v1.1 publish.yml changes
@tusharchou tusharchou self-assigned this Nov 4, 2024
@tusharchou tusharchou added the good first review Good for newcomers label Nov 4, 2024
Copy link
Owner Author

@tusharchou tusharchou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo

  • add following to git ignore
  • .whl
  • .zip
  • .gz

* Release v1.1 dist changes

* Release v1.1 publish.yml changes

* Release v1.1 publish.yml changes

* Release v1.1
@mrutunjay-kinagi
Copy link
Collaborator

@tusharchou Please resolve this conflict.
Run the following commands :

  1. git fetch origin
  2. git switch main
  3. git pull
  4. git switch <branch_with_conflict>
  5. git merge main
  6. Resolve conflicts and commit

@@ -39,7 +39,7 @@ class Table(Base):

def __init__(self, name: str, path: Path = os.getcwd()):
self.name = name
self.path = path
self.path = os.getcwd()+path
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why you have taken os.getcwd() twice, once as argument and other time concatenating ?
pls check

* fixed bug warehouse uri

Thu Oct 24 9:25 PM IST

* added logging to CSV.get()

* supported big query ts format

* refactored parameter to config from catalog to reduce confusion

* Fixed bug of logger in GCP

* replaced local path with dynamic path

* replaced local path with dynamic path

* replaced local path with dynamic path

* demo

* demo

* Release v1.1 dist changes

* Release v1.1 publish.yml changes

* Release v1.1 publish.yml changes

* added a class implementation for github issue hoping it is useful for guiding users to resolution ETAs eventually

* added a exception for PlanNotFound to ask users to raise issues on the repository for resolution

* Updated overview and milestones. Added directory structure under technical specifications.

* Updated components in technical speciifcations

* added testing for BigQueryToCSV.extract()

* added testing for BigQueryToCSV.extract()

* added testing for Iceberg.get()

* Release v1.1

* Release v1.1 bug fix

* Release v1.1 bug fix

* Release v1.1 bug fix

* Pytest Added for BigQuery Source

---------

Co-authored-by: Tushar Choudhary <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment