Determining the byte range for the value of a particular object key

[Aaron1011]

I'm trying to use actson to extract a value from a stream of bytes that looks like:

{"irrelevant-key": "irrelevant-value",      "important-key":   {"something": "important"}}

My goal is to be able to find important-key in the top-level object, and determine which bytes in my stream correspond to the value. Concretely, I want to be able to write just those bytes to an output stream as I receive new bytes in the input stream, without needing to wait for the full value to be read in. In the above example, this would mean writing {"something": "important"} to the output stream (without the whitespace or colon after important-key.

This is almost possible with the current API - I can detect important-key with JsonEvent::FieldName and current_str, and use parsed_bytes to determine where important-key ends. However, there doesn't seem to be a way to detect the start of an arbitrary value (while excluding the colon and whitespace) - if my object contains "important_key": "string_value" or "important_key": 123, I can't determine the start byte of the value, as I'll only get a JsonEvent::Value* event once the entire value has been fed into the parser.

I had a few ideas for an API that would allow extracting this information:

• Add a JsonEvent::StartValue event, which would indicate when the first byte of an object value has been read (e.g. a {, [, ", or leading digit of a number). However, this would mean emitting multiple events for the same byte (as it could be immediately followed by a JsonEvent::StartObject or JsonEvent::StartArray)
• Add JsonEvent::Start* events for the first byte of other values (e.g. JsonEvent::StartTrue when a t byte is read, JsonEvent::StartString when a " byte is read). Unlike the other events, these wouldn't necessarily correspond to a valid value (e.g. the stream {"my-key": tabcd} would produce StartObject, FieldName, StartTrue, <error> since the t byte was not part of the sequence true).

Thanks for making such a useful crate!

[michel-kraemer]

Hi @Aaron1011! Thanks for starting this discussion. Your ideas sound reasonable but before we change anything in the API, I wanted to make sure we've considered all other possibilities.

For example, would something like the following already be enough for you? I know it's not quite what you had in mind, but maybe good enough for a workaround?

let json = r#"{"irrelevant-key": "irrelevant-value",      "important-key":   {"something": "important"}}"#;
let buf = json.as_bytes();

let mut parser = JsonParser::new(PushJsonFeeder::new());
let mut i = 0;
let mut prev_event_end = 0;
while let Some(event) = parser.next_event().unwrap() {
    match event {
        JsonEvent::NeedMoreInput => {
            i += parser.feeder.push_bytes(&buf[i..]);
            if i == json.len() {
                parser.feeder.done();
            }
        }

        JsonEvent::FieldName => {
            if let Ok("important-key") = parser.current_str() {
                println!("Start: {}, end: {}", prev_event_end, parser.parsed_bytes());
            }
        }

        _ => {}
    }
    prev_event_end = parser.parsed_bytes();
}

[Aaron1011]

Hi @michel-kraemer: That example prints a start/end range corresponding to the substring: , "important-key"
Unfortunately, that doesn't include the : token (or any preceding/trailing whitespace), so I'm not able to determine when the start of {"something": "important"} occurs in the input byte stream.

In my use-case, that would cause my application to emit invalid json (e.g. : {"something": "important"}).

[michel-kraemer]

Hi @Aaron1011! Sorry for my late reply. I was trying to find a solution for your issue but I think the required API changes would be too disruptive and might even impact the parser's performance.

Based on my earlier snippet, I came up with another solution instead:

let json = "{\"irrelevant-key\": \"irrelevant-value\",      \
    \"important-key\":   {\"something\": \"important\"},  \
    \"important-key2\"  :   123  ,\
    \"another-irrelevant-key\": 456,\
    \"important-key3\"    :   \"string_value\"  }";
let buf = json.as_bytes();

let mut parser = JsonParser::new(PushJsonFeeder::new());
let mut i = 0;
let mut nested_start = 0;
let mut level = 0;
let mut print_next_value = false;
while let Some(event) = parser.next_event().unwrap() {
    match event {
        JsonEvent::NeedMoreInput => {
            i += parser.feeder.push_bytes(&buf[i..]);
            if i == json.len() {
                parser.feeder.done();
            }
        }
        JsonEvent::FieldName => {
            if let Ok(name) = parser.current_str() {
                if name.starts_with("important-key") {
                    print_next_value = true;
                }
            }
        }

        JsonEvent::ValueString => {
            if print_next_value && level == 0 {
                println!(
                    "{}",
                    parser.current_str().expect("There should be a string")
                );
                print_next_value = false;
            }
        }

        JsonEvent::ValueInt => {
            if print_next_value && level == 0 {
                println!(
                    "{}",
                    parser.current_int::<i64>().expect("There should be an int")
                );
                print_next_value = false;
            }
        }

        JsonEvent::ValueFloat => {
            if print_next_value && level == 0 {
                println!(
                    "{}",
                    parser.current_float().expect("There should be a float")
                );
                print_next_value = false;
            }
        }

        JsonEvent::ValueTrue => {
            if print_next_value && level == 0 {
                println!("true");
                print_next_value = false;
            }
        }

        JsonEvent::ValueFalse => {
            if print_next_value && level == 0 {
                println!("false");
                print_next_value = false;
            }
        }

        JsonEvent::ValueNull => {
            if print_next_value && level == 0 {
                println!("null");
                print_next_value = false;
            }
        }

        JsonEvent::StartObject | JsonEvent::StartArray => {
            if print_next_value {
                if level == 0 {
                    nested_start = parser.parsed_bytes() - 1;
                }
                level += 1;
            }
        }

        JsonEvent::EndObject | JsonEvent::EndArray => {
            if print_next_value {
                level -= 1;
                if level == 0 {
                    println!("{}", &json[nested_start..parser.parsed_bytes()]);
                }
                print_next_value = false;
            }
        }
    }
}

This code correctly prints:

{"something": "important"}
123
string_value

What do you think about it?