[Feature] Add "Wait for completion" option to "GIS file input" (and output) transform

[sfkeller]

In a pipeline I want to integrate a system call using "Execute a process" e.g. to the converter OGR/GDAL - a CLI executable - in order to extend the file format input (and later on the output) capabilities of Hop.

Currently "GIS file input" transform (plugin) doesn't wait intil it starts. Same applies probably also for the "GIS File output" transform (plugin).

The suggestion is, that there is an additional option in "GIS file input" which indicates the transform to wait until completion of the precedent transform (eventually this option isn't even needed if the GIS file input plugin can check that there's the executed process completed successfully).

This works e.g. for the "CVS file input" reader, as shown in the pipeline below, which wait's until OGR (or gdal vector convert) has written a temporary CSV file (which contains the geometry as field WKT of type string).

Goal is, that this also works for "GIS file input" instead of "CSV file input".

Note that this quickfix of the "GIS file input" plugin would not only work for OGR/GDAL, but for any CLI executable which produces a temporary file.

Here are the used files and the ogr2ogr commands for the showcase:

ogr2ogr -f CSV test_output.csv test_input.geojson -lco GEOMETRY=AS_WKT
or
gdal vector convert -f CSV test_output.csv test_input.geojson -lco GEOMETRY=AS_WKT

Tip: GDAL/OGR is always installed with the Windows installer of QGIS and with the official package version App Bundle of QGIS for macOS. For Linux distributions of QGIS, GDAL/OGR must be installed separately.

Alternatives:

Currently no - except to either extend the GIS plugin "GIS file input" (using GeoTools?), or implement a standalone transform based on an OGR/GDAL Java wrapper as described in #28.

NOTE: There is the “Blocking until transform finish” Transform which could be placed in front of the GIS File input plugin, but that doesn't seem to work.

Has the feature been requested before?

Not AFAIK (although this is IMHO an obvious feature request)

If the feature request is approved, would you be willing to submit a PR?

I think this extension would be rather done by a maintainer of the GIS file input transform.
We are willing to contribute in #28.

Files:

• test_output.csv
• test_input.txt (should have file extension .geojson)

[sfkeller]

@cvagner @bamaer @mattcasters What do you think about this issue?

Sorry for pushing. I'm advising a student thesis in the current winter semester and I'm preparing a talk for the next Apache Hop User Meeting (german+english).

[sfkeller]

We're working on a patch and pull request of this issue.