Skip to content

[BUG] | Not able to Read compressed csv file without passing any size limit #34435

Open
@shipragoyal07

Description

@shipragoyal07

Not able to Read compressed csv file without passing any size limit.
Error: '<' not supported between instances of 'int' and 'NoneType'

Language: Python
Version: 3.11
Apache beam version: 2.62.0

Code block:
from apache_beam.io.filesystems import FileSystems with FileSystems.open(gcs_file_uri, 'rb') as file: raw_content = file.read()

where gcs_file_uri is GCS path of the file

Error function: _fetch_to_internal_buffer in module: https://beam.apache.org/releases/pydoc/current/_modules/apache_beam/io/filesystem.html
Line: while not self._read_eof and (self._read_buffer.tell() -
self._read_position) < num_bytes:

where while passing nothing in file.read(), the condition is comparing 0 < None which is uncomparable and hence raising exception.
When passing file.read(10000), its comparing 0<0 which is satisfying the condition and hence working fine.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions