Skip to content

Conversation

@matthewkrausse
Copy link
Collaborator

Introduce functionality for reading and writing Avro files in the Table class, including methods for basic operations, schema handling, and support for various compression codecs.

@matthewkrausse
Copy link
Collaborator Author

Looks like I am running into an issue with the limited dependencies. fastavro should be a dependency of the petl package but they are not being loaded in I guess. Wanted to chat through this before just adding fastavro to the limited dependency set which seems wrong. @austinweisgrau Have any thoughts here?

@austinweisgrau
Copy link
Collaborator

For the limited dependencies setup, you need to add the dependency in the setup.py file. You'll want to think about if it should be an "extra" (so if people want to be able to use that functionality, they need to install parsons with parsons[avro] or if it should auto-bundled with parsons for everyone, even in the friendly dependencies setup.

By default, it should probably be the former (need to opt-in with parsons[avro]).

@matthewkrausse
Copy link
Collaborator Author

Thanks. Made what I believe will fix it. Let's find out.

@github-actions
Copy link

github-actions bot commented Mar 25, 2025

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  parsons/etl
  tofrom.py 218
Project Total  

This report was generated by python-coverage-comment-action

@austinweisgrau
Copy link
Collaborator

Thanks. Made what I believe will fix it. Let's find out.

yep that's it!

@matthewkrausse
Copy link
Collaborator Author

@shaunagm This is ready for your review

@matthewkrausse matthewkrausse requested a review from shaunagm March 25, 2025 21:12
@matthewkrausse matthewkrausse changed the title Add Avro file support in Table class Add Avro file support for Parsons Tables Mar 26, 2025
Copy link
Collaborator

@shaunagm shaunagm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple minor docs changes requested but other than that this looks good

@matthewkrausse matthewkrausse requested a review from shaunagm March 30, 2025 01:24
@shaunagm
Copy link
Collaborator

shaunagm commented Apr 4, 2025

@matthewkrausse there's still a couple links without urls (to documentation and reference) other than that we're good to go

Copy link
Collaborator

@shaunagm shaunagm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just updating the status to reflect that there's still a couple changes needed (a few links without urls still)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants