Skip to content

[ISSUE] warehouses.list() silently drops warehouse_type for warehouse types not in the SDK enum #1484

Description

@phongl7

Description

WorkspaceClient.warehouses.list() (and .get()) returns EndpointInfo objects whose warehouse_type is missing entirely when the server returns a warehouse type value that isn't one of the three the SDK's enum knows about. We hit this with the real-time SQL warehouse type (REAL_TIME): the REST API returns a warehouse_type, but warehouse.as_dict() has no warehouse_type key at all.

This is silent data loss — the field the server sent just disappears, with no error or warning.

Reproduction

from databricks.sdk.service.sql import EndpointInfo

# What the REST API returns for the real-time warehouse type (newer than the SDK's enum):
ei = EndpointInfo.from_dict({"id": "abc", "name": "shared-real-time", "warehouse_type": "REAL_TIME"})

print(ei.warehouse_type)            # -> None
print("warehouse_type" in ei.as_dict())   # -> False   (field is gone)

Root cause

EndpointInfoWarehouseType only defines CLASSIC, PRO, TYPE_UNSPECIFIED
(databricks/sdk/service/sql.py). Deserialization goes through _enum()
(databricks/sdk/service/_internal.py):

def _enum(d, field, cls):
    """Unknown enum values are returned as None."""
    ...
    return _get_enum_value(cls, d[field])   # returns None for any unknown value

as_dict() then omits None fields (if self.warehouse_type is not None), so an
unknown enum value is lost completely — not even the raw string survives.

There are really two problems:

  1. Missing enum valueEndpointInfoWarehouseType (and GetWarehouseResponseWarehouseType) need REAL_TIME. Since sql.py is code-generated (DO NOT EDIT), this has to be fixed in the upstream OpenAPI spec.
  2. Silent data loss for any unknown enum — even once the spec catches up, the SDK will keep dropping any future server-side enum value it doesn't recognize. This affects every enum field SDK-wide, not just warehouses.

Expected behavior

Unknown enum values returned by the server should be preserved, not silently dropped, so that as_dict() reflects what the API actually returned.

Proposed fix

For (2), preserve unknown values via a small wrapper in _internal.py that still
round-trips through as_dict() (which calls .value), while remaining
distinguishable from a known enum member.
PR: #1485

For (1), the enum value needs to be added in the spec.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions