How to Skip Parsing Large Objects in a JSON String Using nlohmann/json

[Cirnoo]

Hi,

I need to parse a large JSON string, for example:

std::string json_data = R"({
        "key1": "value1",
        "key2": "value2",
        "largeObject": {
            "nestedKey": "nested_value",
            "largeSubObject": {
                "someKey":"someValue"
            }
        },
        "key3": "value3"
    })";

Considering performance, I don’t want to parse the entire largeObject at once. Instead, I would like to convert largeObject to a string to avoid fully parsing it. How can I implement this? Thanks a lot.

Here is an example of what the code might look like:

std::string json_data = R"({
        "key1": "value1",
        "key2": "value2",
        "largeObject": {
            "nestedKey": "nested_value",
            "largeSubObject": {
                "someKey":"someValue"
            }
        },
        "key3": "value3"
    })";


    
    json parsed_json = myParse(json_data);

    printf("data=%s\n", parsed_json.dump().data());

Expected output:

{
        "key1": "value1",
        "key2": "value2",
        "largeObject": "{\"nestedKey\":\"nested_value\",\"largeSubObject\":{\"someKey\":\"someValue\"}}"  // been parsed to jsonstring intead of jsonobject
        "key3": "value3"
}

[nlohmann]

You could try https://json.nlohmann.me/features/parsing/parser_callbacks/ or https://json.nlohmann.me/features/parsing/sax_interface/

[Cirnoo]

I have tried using parser_callbacks or sax_interface, and they indeed allow for skipping large objects. However, I would like to retain the raw string of the skipped large object.

When using sax_interface, returning false from the function interrupts the entire parse process. with parser_callbacks, I am unable to retrieve the complete raw string before parsing the large object.

I would greatly appreciate any advice or suggestions you may have! Thank you very much.
Here is my code:

#include <iostream>
#include <pie/json.hpp>


bool my_callback(int depth, nlohmann::json::parse_event_t event, nlohmann::json &parsed)
{
    switch (event) {
        case nlohmann::json::parse_event_t::object_start:
            // TODO? I can't get raw 'large object string' from the input;
            std::cout << std::string(depth, ' ') << "start object" << std::endl;
        break;
        case nlohmann::json::parse_event_t::object_end:
           
        break;
        case nlohmann::json::parse_event_t::array_start:
            
        break;
        case nlohmann::json::parse_event_t::array_end:
            
        break;
        case nlohmann::json::parse_event_t::key:
            std::cout << std::string(depth, ' ') << "key: "  << std::endl;
        break;
        case nlohmann::json::parse_event_t::value:
           
        break;
    }
    return true;
}

int main()
{
    std::string json_data = R"(
    {
        "largeObject": {
            "nestedKey": "nested_value",
            "largeSubObject": {
                "someKey":"someValue"
            }
        }
    })";

    nlohmann::json parsed_json = nlohmann::json::parse(json_data, my_callback, true);

    std::string large_data = parsed_json["largeObject"];
    // I only need to retrieve the raw string of the large object, as parsing it is a resource-intensive task

    return 0;
}

[nlohmann]

There is currently no way to access the raw input in later parsing stages.