Why do pkgconfig files exist without corresponding CMake modules?

[WillAyd]

In #6285 it appears that we added .pc files for things like arrow-csv, arrow-json, etc... My best guess is that this was done to signal for a given Arrow installation, if it was built with support for those options. However, it doesn't appear that there is a CMake equivalent (?) so its hard to rely on that mechanism universally.

Taking Meson as an example, its trivial to see if the host system provides arrow-compute, whether through pkgconfig or through CMake's own discovery:

arrow_compute_dep = dependency(
    'arrow-compute',
    'ArrowCompute',
    modules: ['ArrowCompute::arrow_compute_shared'],
)

but for something like arrow-csv there is no equivalent CMake discovery (?)

arrow_csv_dep = dependency(
    'arrow-csv',  # works fine if arrow-csv.pc exists
    'ArrowCsv',  # not a real thing - info unavailable for CMake installation
)

[pitrou]

@kou

[kou]

I didn't imagine Meson use case when I created it...

In CMake, we can use the following to check CSV availability:

find_package(Arrow REQUIRED)
if(NOT ARROW_CSV)
  message(FATAL_ERROR "CSV must be enabled")
endif()

See also how Apache Arrow C GLib does:

arrow/c_glib/meson.build

Lines 101 to 142 in 5a48044

	common_args = {'version': [f'>=@version_no_snapshot@']}
	arrow = dependency(
	'arrow',
	'Arrow',
	kwargs: common_args,
	modules: ['Arrow::arrow_shared'],
	)
	# They are just for checking required modules are enabled. They are built into
	# libarrow.so. So we don't need additional build flags for them.
	if arrow.type_name() == 'cmake'
	assert(
	arrow.get_variable('ARROW_COMPUTE', default_value: 'OFF') == 'ON',
	'compute module must be enabled',
	)
	assert(
	arrow.get_variable('ARROW_CSV', default_value: 'OFF') == 'ON',
	'CSV module must be enabled',
	)
	assert(
	arrow.get_variable('ARROW_FILESYSTEM', default_value: 'OFF') == 'ON',
	'filesystem module must be enabled',
	)
	assert(
	arrow.get_variable('ARROW_JSON', default_value: 'OFF') == 'ON',
	'JSON module must be enabled',
	)
	else
	dependency('arrow-compute', kwargs: common_args)
	dependency('arrow-csv', kwargs: common_args)
	dependency('arrow-filesystem', kwargs: common_args)
	dependency('arrow-json', kwargs: common_args)
	endif
	if arrow.type_name() == 'cmake'
	have_arrow_orc = (arrow.get_variable('ARROW_ORC', default_value: 'OFF') == 'ON')
	else
	have_arrow_orc = dependency(
	'arrow-orc',
	kwargs: common_args,
	required: false,
	).found()
	endif

If we want to provide the same interface for pkg-config and CMake package, I suggest that we add variables to arrow.pc something like the following:

diff --git a/cpp/src/arrow/arrow.pc.in b/cpp/src/arrow/arrow.pc.in
index 309789379a..27d2899ccd 100644
--- a/cpp/src/arrow/arrow.pc.in
+++ b/cpp/src/arrow/arrow.pc.in
@@ -23,6 +23,8 @@ so_version=@ARROW_SO_VERSION@
 abi_version=@ARROW_SO_VERSION@
 full_so_version=@ARROW_FULL_SO_VERSION@
 
+csv=@ARROW_CSV@
+
 Name: Apache Arrow
 Description: Arrow is a set of technologies that enable big-data systems to process and move data fast.
 Version: @ARROW_VERSION@

With the approach, we can use the following Meson configuration:

assert(
    arrow.get_variable(cmake: 'ARROW_CSV', pkgconfig: 'csv', default_value: 'OFF') == 'ON',
    'CSV module must be enabled',
)

In pkg-config context, pkgconf --exists arrow-csv is easier use than [ "$(pkgconf --variable csv arrow)" = "ON" ]. So we should not remove arrow-csv with the approach. The approach is just for CMake package compatibility.

[WillAyd]

If we want to provide the same interface for pkg-config and CMake package, I suggest that we add variables to arrow.pc something like the following:

Seems reasonable. Would there be any merit to an alternative of adding FindArrowCSV.cmake, FindArrowJSON.cmake, etc...? Without having to think about Arrow packaging "internals," my thinking is that its consistent to just say "if you can find the package, its enabled" regardless of if you use pkg-conf or cmake

[kou]

We can't provide meaningful CMake target for ArrowCSV and so on because they don't need additional libraries nor build flags.

We can provide ArrowCSVConfig.cmake and define the following useless CMake target in ArrowCSVConfig.cmake:

find_dependency(Arrow)
if(ARROW_BUILD_SHARED)
  add_library(Arrow::arrow_csv_shared INTERFACE IMPORTED)
  target_link_libraries(Arrow::arrow_csv_shared INTERFACE Arrow::arrow_shared)
endif()
if(ARROW_BUILD_STATIC)
  add_library(Arrow::arrow_csv_static INTERFACE IMPORTED)
  target_link_libraries(Arrow::arrow_csv_static INTERFACE Arrow::arrow_static)
endif()

I don't have a strong opinion which approach is better. We can provide one of them or both of them.

[pitrou]

I don't know, but at some point we might want to have a dedicated libarrow_csv.so.