Merged
Conversation
RobertSamoilescu
force-pushed
the
fix/pandas-codec
branch
from
fd72229 to
a9c2289
Compare
RobertSamoilescu
force-pushed
the
fix/pandas-codec
branch
from
7b13ad6 to
1c9de51
Compare
RobertSamoilescu
force-pushed
the
fix/pandas-codec
branch
from
e380a28 to
c793595
Compare
RobertSamoilescu
force-pushed
the
fix/pandas-codec
branch
from
1cb918b to
f98d086
Compare
RobertSamoilescu
force-pushed
the
fix/pandas-codec
branch
from
4d384cc to
041b475
Compare
RobertSamoilescu
added a commit
that referenced
this pull request
Apr 11, 2025
* Implemented JSONCodec * Encode object type with JSONCodec for pandas * Included more types for mlflow * Implemented can_encode function for JSON codec * Included test for JSON codec * Fixed pandas codec to treat objects differently * Included tests for mlflow metadata * Included test for mlflow runtime * Fixed can_encode for JSONCodec and a pandas test * Fixed code registry but need to remove duplicated json encoding code from hf runtime * Fixed linting issues * Refactored duplicated code and simplified logic for can_encode * Renamed util function * Fixed support to encode numpy and reverted can_encode function * Fixed bug in pandas encoding * Reverted comment in pandas codec * Moved env and cli tests to sequential. * Included sleep before retrieving metrics * Run metrics tests seq * Simplified code by removing JSONCodec * Included more tests for pandas codecs * Moved PandasJsonContentType inside PandasCodec.
RobertSamoilescu
added a commit
that referenced
this pull request
Apr 11, 2025
* Implemented JSONCodec * Encode object type with JSONCodec for pandas * Included more types for mlflow * Implemented can_encode function for JSON codec * Included test for JSON codec * Fixed pandas codec to treat objects differently * Included tests for mlflow metadata * Included test for mlflow runtime * Fixed can_encode for JSONCodec and a pandas test * Fixed code registry but need to remove duplicated json encoding code from hf runtime * Fixed linting issues * Refactored duplicated code and simplified logic for can_encode * Renamed util function * Fixed support to encode numpy and reverted can_encode function * Fixed bug in pandas encoding * Reverted comment in pandas codec * Moved env and cli tests to sequential. * Included sleep before retrieving metrics * Run metrics tests seq * Simplified code by removing JSONCodec * Included more tests for pandas codecs * Moved PandasJsonContentType inside PandasCodec.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PandasCodecimproperly encodes columns of numeric lists. The type returned by the codec isBYTESwhile the content still remains a list of numerical values. This leads to improper internal decoding in SC2 (see function here), which leads to a panic due to improper conversion from numeric (e.g., float) to bytes.This PR fixes the encoding object pandas columns to str/bytes which properly match the type of the
Response/Request. This will lead to proper decoding within SC2.In addition, this PR adds support for additional column types supported by the mlflow runtime:
Array,Map,Object,Any. Note that all of they are Json-convertible objects which justifies the choice of encoding them as json.Note that defining a dedicated
JSONCodecmight not be possible because it can clash with theHuggingfaceListJSONCodecorHuggingfaceJSONCodec- multiple codecs can be found based on thecan_encodemethod.