Skip to content

SNOW-280650: CopyIntoStorage will not load data from an ExternalStage #213

Closed as not planned
@ajmarks

Description

@ajmarks

Please answer these questions before submitting your issue. Thanks!

  1. What version of Python are you using (python --version)?
    Python 3.8.5

  2. What operating system and processor architecture are you using (python -c 'import platform; print(platform.platform())')?
    Windows-10-10.0.19041-SP0

  3. What are the component versions in the environment (pip list)?

...
snowflake-connector-python 2.3.9            py38haa244fe_0    conda-forge
snowflake-sqlalchemy      1.2.4              pyh9f0ad1d_0    conda-forge
...
  1. What did you do?
    Attempt to COPY INTO from an external stage
from snowflake.sqlalchemy import CopyIntoStorage, ExternalStage

qry = CopyIntoStorage(
    into=table,
    from_=ExternalStage(stage_name),
    formatter=sa.text(format_spec),
)
conn.execute(qry)
  1. What did you expect to see?
    A working query. Instead, the compiler wraps the stage name in parens, so the query is, e.g.,
    COPY INTO public.test_table_65234d02 FROM (@my_stage) ... instead of COPY INTO public.test_table_65234d02 FROM @my_stage.

The issue is in base.py:190:

        elif isinstance(copy_into.from_, AWSBucket) or isinstance(copy_into.from_, AzureContainer):
            from_ = copy_into.from_._compiler_dispatch(self, **kw)
        # everything else (selects, etc.)
        else:
            from_ = '({})'.format(copy_into.from_._compiler_dispatch(self,
                                                                     **kw))

This should be

        elif isinstance(copy_into.from_, (AWSBucket, AzureContainer, ExternalStage)):
            from_ = copy_into.from_._compiler_dispatch(self, **kw)
        # everything else (selects, etc.)
        else:
            from_ = '({})'.format(copy_into.from_._compiler_dispatch(self,
                                                                     **kw))

Even better would be to have all three inherit from a common ancestor and check on that so that it can be easily extended to include GCP or other providers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions