Skip to content

'open_images_v4' with 'array_record' raise ValueError #5419

Open
@seongmoon729

Description

/!\ PLEASE INCLUDE THE FULL STACKTRACE AND CODE SNIPPET

Short description
When I try to load the 'open_images_v4' dataset with 'array_record' file format, it raises ValueError.

Environment information

  • Operating System: ubuntu 20.04

  • Python version: 3.8.13

  • tensorflow-datasets/tfds-nightly version: tensorflow-datasets=4.9.2

  • tensorflow/tf-nightly version: no tensorflow

  • Does the issue still exists with the last tfds-nightly package (pip install --upgrade tfds-nightly) ? Yes, I checked with tfds-nightly-4.9.2.dev202308090034.

Reproduction instructions

import os
os.environ.pop('TFDS_DATA_DIR', None)
import tensorflow_datasets as tfds

builder = tfds.builder('open_images_v4', file_format='array_record', data_dir='./tensorflow_datasets/')
builder.download_and_prepare()
ds = builder.as_data_source()['train']
print(ds[0])

Link to logs
Traceback (most recent call last):
File "test_tfds.py", line 27, in
batch = next(ds)
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/data_sources/array_record.py", line 83, in iter
yield self[i]
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/data_sources/array_record.py", line 95, in getitem
return self.dataset_info.features.deserialize_example_np(
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/features/top_level_feature.py", line 159, in deserialize_example_np
return self.decode_example_np(example_data, decoders=decoders)
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/features/top_level_feature.py", line 112, in decode_example_np
return self._decode_example_generic(
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/features/top_level_feature.py", line 81, in _decode_example_generic
flatten_decoded = [
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/features/top_level_feature.py", line 82, in
_decode_feature( # pylint: disable=g-complex-comprehension
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/features/top_level_feature.py", line 209, in _decode_feature
return decoder.decode_example_np(example, **decode_kwargs)
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/features/tensor_feature.py", line 234, in decode_example_np
return example_data.reshape(shape)
ValueError: cannot reshape array of size 8 into shape (4,)

Expected behavior
Load the 'open_images_v4' dataset without exception and print a data

Additional context
Add any other context about the problem here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions