Skip to content

JSONAttribute(null=True) field TypeError: the JSON object must be str, bytes or bytearray, not 'NoneType'  #627

Open
@monkut

Description

@monkut

pynamodb 3.3.3
Local testing done with Localstack: image: localstack/localstack

I have the following model:

class CollectionStateIndex(GlobalSecondaryIndex):

    class Meta:
        index_name = settings.DYNAMODB_COLLECTION_STATE_INDEXNAME
        read_capacity_units = 2
        write_capacity_units = 1

        projection = AllProjection()
        host = settings.DYNAMODB_ENDPOINT

    collection_id = UnicodeAttribute(hash_key=True)
    state = UnicodeAttribute(range_key=True)


class S3ItemModel(Model):
    s3_uri = UnicodeAttribute()
    collection_id = UnicodeAttribute()
    request_id = UnicodeAttribute(hash_key=True)
    state = UnicodeAttribute()
    created_at_timestamp = NumberAttribute()
    updated_at_timestamp = NumberAttribute()
    result = JSONAttribute(null=True)
    errors = JSONAttribute(null=True)

    collections_state_index = CollectionStateIndex()

    class Meta:
        table_name = settings.DYNAMODB_S3ITEMMODEL_TABLENAME
        region = settings.AWS_REGION
        host = settings.DYNAMODB_ENDPOINT
        read_capacity_units = 2
        write_capacity_units = 1

And it appears that when performing a query for a result where the errors value is Null, the query fails to properly decode the item with the exception:

TypeError: the JSON object must be str, bytes or bytearray, not 'NoneType'

item = next(results)
File "/var/task/pynamodb/pagination.py", line 183, in __next__
item = self._map_fn(item)
File "/var/task/pynamodb/models.py", line 529, in from_raw_data
kwargs[attr_name] = attr.deserialize(attr.get_value(value))
File "/var/task/pynamodb/attributes.py", line 439, in deserialize
return json.loads(value, strict=False)
File "/var/lang/lib/python3.6/json/__init__.py", line 348, in loads
'not {!r}'.format(s.__class__.__name__))
TypeError: the JSON object must be str, bytes or bytearray, not 'NoneType'

Query:

        results = S3ItemModel.query(
            hash_key=request_id,
        )
        item = next(results)

DynamoDB Record:

{
  "collection_id": {
    "S": "collection:11"
  },
  "created_at_timestamp": {
    "N": "1558593743"
  },
  "errors": {
    "NULL": true
  },
  "request_id": {
    "S": "a0cddfe0-aa9f-5dex-88b5-bf808f249edc"
  },
  "result": {
    "S": "[]"
  },
  "s3_uri": {
    "S": "s3://bucketname/key"
  },
  "state": {
    "S": "processed"
  },
  "updated_at_timestamp": {
    "N": "1558593743"
  }
}

Why is deserialization failing for the errors field?
With JSONAttribute(null=True) shouldn't it be able to handle null values for the errors field?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions