GH-45908: [C++][Docs] Rename and expose basic {Array,...}FromJSON helpers as public APIs #46180

amoeba · 2025-04-18T18:37:17Z

Rationale for this change

These functions are generally useful and stable so it would be a good idea to clearly include them in the public API. I'm starting here with just the basic ones in order to make the PR small. See #45908 for more information.

What changes are included in this PR?

Moves ArrayFromJSON, ChunkedArrayFromJSON, DictArrayFromJSON, ScalarFromJSON, DictScalarFromJSON from arrow::ipc::internal namespace to arrow::json so it's clearer they're part of the public API and that they're more useful than just for IPC
Renames each of the above from {Array,...}FromJSON to {Array,...}FromJSONString to avoid confusion between these helpers and the main JSON(L) reader
Renames arrow/util/json_simple.{h,cc} to arrow/json/from_string.{h,cc} both because of the namespace jump but also because the filename is more clear.
Adds User Guide and adds a listing in the API docs for moved functions

Are these changes tested?

Yes.

Are there any user-facing changes?

This expands the scope of our public API but does not break any existing public APIs though it's possible users

GitHub Issue: [C++] Expose {Array,...}FromJSON as public APIs #45908

This moves the following functions from the IPC namespace to util to make it clear these are useful outside of their use in Arrow IPC. - ArrayFromJSON - ChunkedArrayFromJSON - DictArrayFromJSON - ScalarFromJSON - DictScalarFromJSON

amoeba · 2025-04-21T02:41:13Z

@github-actions crossbow submit preview-docs

github-actions · 2025-04-21T02:43:33Z

Revision: cf808f9

Submitted crossbow builds: ursacomputing/crossbow @ actions-7a9d15701e

Task	Status
preview-docs

amoeba · 2025-04-21T03:27:46Z

Docs previews:

zanmato1984

Thank you for bringing this up! I have some minor comments.

cpp/src/arrow/CMakeLists.txt

cpp/src/arrow/acero/hash_join_node_test.cc

cpp/src/arrow/meson.build

docs/source/cpp/api/array.rst

github-actions · 2025-05-08T14:46:58Z

Revision: 5cedcfe

Submitted crossbow builds: ursacomputing/crossbow @ actions-b687d19c46

Task	Status
preview-docs

bkietz · 2025-05-08T15:00:32Z

I would guess this failure is caused by use of é (e with a diacritic). The bytes of a string literal aren't guaranteed to be utf-8 encoded without a u8 or u8R prefix/lex.string#7, like u8"héhé". We will probably need to revisit that

amoeba · 2025-05-08T16:17:46Z

Interesting. It would be nice to know what it is about that particular job that makes it fail but I can file a minor PR for it since it sounds like a fix we should make.

amoeba · 2025-05-08T16:26:33Z

After the latest round of reviews, I noticed some errors with my inline example C++ code and realized it was probably better to write an actual example source file and literalinclude that from the docs. See 5cedcfe.

Latest docs preview

Can you look again @bkietz? Thanks.

bkietz · 2025-05-08T18:31:05Z

It would be nice to know what it is about that particular job that makes it fail but I can file a minor PR for it since it sounds like a fix we should make.

The rest of the PR seems ready to go, for now could you comment out any cases in the GDB test which reference utf-8 strings to verify that's the only thing failing? (And keep CI green until you make the minor PR; I'll help review)

bkietz · 2025-05-08T20:25:33Z

Well, those cases appear to have been responsible for that failure. After looking more closely, we've got an issue open for this already: #46343

This reverts commit b5ed9b1.

amoeba · 2025-05-08T21:04:55Z

Nice catch. I commented there. I reverted the testing commit so the PR is in a merge-able state. Would you be okay if we merged with that one job failing or should we wait on #46343 and rebase before merge?

EnricoMi

Some minor comments.

cpp/examples/arrow/from_json_string_example.cc

cpp/src/arrow/testing/gtest_util.cc

Co-authored-by: Enrico Minack <[email protected]>

amoeba · 2025-05-09T17:59:38Z

Thanks @bkietz. There were a couple of comments from @EnricoMi and I'll merge this once those are resolved.

EnricoMi

LGTM!

amoeba · 2025-05-14T14:23:00Z

Merged. Thanks all for the reviews, it's greatly appreciated.

pitrou · 2025-05-14T14:33:33Z

cpp/src/arrow/dataset/test_util_internal.h

@@ -2140,8 +2140,8 @@ class WriteFileSystemDatasetMixin : public MakeFileSystemDatasetMixin {
        actual_struct = std::dynamic_pointer_cast<Array>(struct_array);
      }

-      auto expected_struct = ArrayFromJSON(struct_(expected_physical_schema_->fields()),
-                                           file_contents->second);
+      auto expected_struct = arrow::ArrayFromJSON(


It's not obvious to me why you needed to add explicit arrow:: prefixes here?

You're right, nice catch. I can remove this. Is there an automated way to catch things like this?

Not to my knowledge, no.

pitrou · 2025-05-14T14:34:38Z

cpp/src/arrow/json/from_string.cc

 namespace json {

 using ::arrow::internal::checked_cast;
 using ::arrow::internal::checked_pointer_cast;

-namespace {
+namespace internal {


Why not keep this in the anonymous namespace as it was?

I made this change to avoid a conflict with our other Converter class. Now that I see what I've done here, maybe it's better just to rename it to give it a better name and keep it in an anonymous namespace?

Yes, it would be better to rename IMHO.

pitrou · 2025-05-14T14:35:09Z

cpp/src/arrow/json/from_string.h

+/// );
+/// \endcode
+ARROW_EXPORT
+Status ChunkedArrayFromJSONString(const std::shared_ptr<DataType>& type,


Why not return a Result<std::shared_ptr<ChunkedArray>> instead of taking an out-pointer parameter?

Yeah, I saw the inconsistency in these and don't love it. I haven't looked yet but it seems like we could make all of these helpers consistent (return Result<std::share_ptr<T>>) instead of using out-params in some and Result in others. I'll take a look now so we don't have to make a breaking change.

This was easy to do, I put up a draft PR and converted ChunkedArrayFromJSONString in ede205e.

I can do the rest if you think that makes sense (I do).

Yes, I think we should convert all of them.

pitrou · 2025-05-14T14:37:20Z

I posted a couple comments, it would be nice to do a post-commit polish PR.

amoeba · 2025-05-14T14:46:20Z

I'll file an issue to follow-up on those.

conbench-apache-arrow · 2025-05-14T21:37:34Z

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 07778d9.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.

This comment was marked as resolved.

Sign in to view

github-actions bot added Component: C++ awaiting review Awaiting review Component: Python Component: Documentation labels Apr 18, 2025

amoeba added 2 commits April 20, 2025 18:00

Move core *FromJSON helpers from ipc to util NS

e6b8267

This moves the following functions from the IPC namespace to util to make it clear these are useful outside of their use in Arrow IPC. - ArrayFromJSON - ChunkedArrayFromJSON - DictArrayFromJSON - ScalarFromJSON - DictScalarFromJSON

Add docs for FromJSON helpers

cf808f9

amoeba force-pushed the feature/GH-45908--expose-from-json branch from 3b6015e to cf808f9 Compare April 21, 2025 01:00

amoeba mentioned this pull request Apr 21, 2025

[C++] Fix ASAN issue in the arrow-dataset-dataset-writer-test #45235

Open

amoeba changed the title ~~GH-45908: [C++] Expose {Array,...}FromJSON as public APIs~~ GH-45908: [C++] Expose basic {Array,...}FromJSON helpers as public APIs Apr 21, 2025

amoeba marked this pull request as ready for review April 21, 2025 02:36

amoeba requested review from AlenkaF, raulcd, rok and westonpace as code owners April 21, 2025 02:36

amoeba requested review from bkietz, zanmato1984 and pitrou April 21, 2025 02:37

amoeba mentioned this pull request Apr 21, 2025

[C++] Expose {Array,...}FromJSON as public APIs #45908

Closed

amoeba changed the title ~~GH-45908: [C++] Expose basic {Array,...}FromJSON helpers as public APIs~~ GH-45908: [C++][Docs] Expose basic {Array,...}FromJSON helpers as public APIs Apr 21, 2025

Update meson.build

644d407

zanmato1984 reviewed Apr 21, 2025

View reviewed changes

cpp/src/arrow/CMakeLists.txt Show resolved Hide resolved

cpp/src/arrow/acero/hash_join_node_test.cc Outdated Show resolved Hide resolved

cpp/src/arrow/meson.build Outdated Show resolved Hide resolved

docs/source/cpp/api/array.rst Outdated Show resolved Hide resolved

github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Apr 21, 2025

amoeba added 3 commits April 21, 2025 14:52

Fix path in meson.build

6fbb2f3

Remove JSON bits from IPC

ce373f5

Use using in hash_join_node_test.cc

6479400

testing theory on ci failure

b5ed9b1

amoeba mentioned this pull request May 8, 2025

[CI][Python] Conda Python 3.10 jobs fail with UnicodeDecodeError due to gdb issue #46343

Open

Revert "testing theory on ci failure"

96ed729

This reverts commit b5ed9b1.

EnricoMi reviewed May 9, 2025

View reviewed changes

bkietz approved these changes May 9, 2025

View reviewed changes

github-actions bot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels May 9, 2025

Update cpp/examples/arrow/from_json_string_example.cc

f91dd1c

Co-authored-by: Enrico Minack <[email protected]>

EnricoMi approved these changes May 10, 2025

View reviewed changes

amoeba merged commit 07778d9 into apache:main May 14, 2025
41 of 42 checks passed

amoeba removed the awaiting merge Awaiting merge label May 14, 2025

pitrou reviewed May 14, 2025

View reviewed changes

github-actions bot added the awaiting committer review Awaiting committer review label May 14, 2025

pitrou reviewed May 14, 2025

View reviewed changes

amoeba mentioned this pull request May 14, 2025

[C++] Address post-merge review comments in PR exposing {Array,...}FromJSON helpers in public API #46439

Open

This was referenced May 16, 2025

Fix Meson Confiugration to Include Recent JSON Changes #46478

Closed

GH-46478: [C++] Implement recent JSON changes into Meson configuration #46479

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GH-45908: [C++][Docs] Rename and expose basic {Array,...}FromJSON helpers as public APIs #46180

GH-45908: [C++][Docs] Rename and expose basic {Array,...}FromJSON helpers as public APIs #46180

amoeba commented Apr 18, 2025 •

edited

Loading

This comment was marked as resolved.

amoeba commented Apr 21, 2025

github-actions bot commented Apr 21, 2025

amoeba commented Apr 21, 2025

zanmato1984 left a comment

github-actions bot commented May 8, 2025

bkietz commented May 8, 2025

amoeba commented May 8, 2025

amoeba commented May 8, 2025

bkietz commented May 8, 2025

bkietz commented May 8, 2025

amoeba commented May 8, 2025

EnricoMi left a comment

amoeba commented May 9, 2025

EnricoMi left a comment

amoeba commented May 14, 2025

pitrou May 14, 2025

amoeba May 14, 2025

pitrou May 15, 2025

pitrou May 14, 2025

amoeba May 14, 2025

pitrou May 15, 2025

pitrou May 14, 2025

amoeba May 14, 2025

amoeba May 14, 2025

pitrou May 15, 2025

pitrou commented May 14, 2025

amoeba commented May 14, 2025

conbench-apache-arrow bot commented May 14, 2025

GH-45908: [C++][Docs] Rename and expose basic {Array,...}FromJSON helpers as public APIs #46180

GH-45908: [C++][Docs] Rename and expose basic {Array,...}FromJSON helpers as public APIs #46180

Conversation

amoeba commented Apr 18, 2025 • edited Loading

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

This comment was marked as resolved.

amoeba commented Apr 21, 2025

github-actions bot commented Apr 21, 2025

amoeba commented Apr 21, 2025

zanmato1984 left a comment

Choose a reason for hiding this comment

github-actions bot commented May 8, 2025

bkietz commented May 8, 2025

amoeba commented May 8, 2025

amoeba commented May 8, 2025

bkietz commented May 8, 2025

bkietz commented May 8, 2025

amoeba commented May 8, 2025

EnricoMi left a comment

Choose a reason for hiding this comment

amoeba commented May 9, 2025

EnricoMi left a comment

Choose a reason for hiding this comment

amoeba commented May 14, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pitrou commented May 14, 2025

amoeba commented May 14, 2025

conbench-apache-arrow bot commented May 14, 2025

amoeba commented Apr 18, 2025 •

edited

Loading