V3#615
Merged
Merged
Conversation
Seems to work better for interactive development Might have problems with the makefile in CI, we'll see
Only really useful for persistence and I'm going to remove that
Should have broken up dev and hdfs oh well
Remove functionality and related tests Coverage still looks good
Apparently I've let that lapse
No more reference to API class, just link to modules
I'm removing a ton of stuff, that's definitely a breaking change
Could have sworn I did this before
Only really made sense with hdfs and other state management
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Remove hdfs depdendencies
Remove local management of data generally
Focus just on API calls and immediately reading downloaded files to pandas
local storage and version management shouldn't be in scope for this tool and removing the heavy dependencies makes it usable more places.
Way back in the day I had users having issues working with it in google drive, and I v2 won't work or even install in Databricks.
In future enhancements I might provide options for different data interfaces besides pandas like spark or polars.
Also switched from poetry to uv because all the cool kids are doing it and it's legitimately faster and easier to work with.