Skip to content

Cannot import values as number/decimal #255

Open
@dluo-sig

Description

@dluo-sig

I'm trying to figure out how to get my data in as a number/decimal, instead of a double. My data type in SQL server is decimal(38,20). Here is my output data type:

"val2":{"inclusion":"available","multipleOf":1e-20,"type":["null","number"]}

However, this seems to come out in snowflake as a double/float and loses the precision. I see that a potential fix for this is to specify singer_decimal on the tap. In that case, I see that the data type is the following:

"val2":{"inclusion":"available","format":"singer.decimal","type":["null","number","string"],"additionalProperties":{"scale_precision":"(38,20)"}}

The problem with this approach is that it ends up in snowflake as a varchar. For what it's worth, regardless of singer_decimal specification, the actual value that I see in the output file of the tap has the correct precision. It's just lost somewhere when pushing it to snowflake.

Not sure if it's related, but I also see this in the code defining the python type as a float:

class NUMBER(sct.NUMBER):
    """Snowflake NUMBER type."""

    def __init__(self, *args: t.Any, **kwargs: t.Any) -> None:
        super().__init__(*args, **kwargs)

    @property
    def python_type(self):
        return float

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions