Error reading the file "edges.geoparquet" in "synthetic_write()", "post_processing.py" (minimum viable configuration file)

[3501744176]

Hello:
I tested SWMManywhere using the "minimum viable configuration file", but encountered a failure when reading the file "edges.geoparquet".

The error occurs in "post_processing.py", within the function "synthetic_write()":

File ".../site-packages/swmmanywhere/post_processing.py", line 40, in synthetic_write
edges = gpd.read_file(addresses.model_paths.edges)
ValueError: setting an array element with a sequence
This issue appears to be related to how edges.geoparquet is written in preprocessing.py by the function write_df().

When I attempt to read the same file using "gpd.read_parquet()" instead, a different error is raised:

---

pyarrow.lib.ArrowInvalid: Parquet magic bytes not found in footer.
Either the file is corrupted or this is not a parquet file.

This suggests that although the file has a .geoparquet extension, it may not be written as a valid Parquet/GeoParquet file, which causes failures when reading it either via read_file() or read_parquet().

Environment:
Python: 3.10
pyarrow: 22.0.0
geopandas: 1.1.2

Please let me know if additional information or a reproducible snippet would be helpful.

[barneydobson]

Hi @3501744176 , thanks for the issue - I can reproduce locally. It looks like some of the file writing/reading with parquet was not working, keeping the file name as parquet, but writing/reading it as json. It seems that this new version of geopandas is (sensibly) stricter on checking extensions of what is written/read.

I will tackle this soonest after the festive period 👍

[barneydobson]

(If you want to use SWMManywhere right now - a quick fix should be to set extension: json in your config.yml

[barneydobson]

Note:

Seems the problem line is in the saving final graph.

But also check all to_file