(WIP) Add benchmarks comparing performance of parse functions

[duckinator]

Additional changes that were needed for benchmarking purposes, but may not be wanted:

• adding pg_query_raw_parse to the bindings
• exposing pg_query::bindings

[duckinator]

Summary

I couldn't find a way to expose pg_query_raw_parse(), so I focused on the rest of the task.

For this query, pg_query_parse_protobuf() seems to take 5x as long as pg_query_parse().

Details

As I discussed with @seanlinsley on Slack, I couldn't find a way to expose pg_query_raw_parse(), so I set that aside since this work is time-boxed.

You can run cargo bench to get benchmark output. Unfortunately, in order to accommodate per-benchmark flamegraph generation, the cargo bench output is a bit messier.

cargo bench output:

     Running benches/parse.rs (target/release/deps/parse-e7cc8bbbe183b46f)
Starting: Running benchmark(s). Stand by!

•

Method                 Mean        Samples
------------------------------------------
pg_query_parse    922.41 μs    2,463/2,500

     Running benches/parse_protobuf.rs (target/release/deps/parse_protobuf-93fcb930536d0376)
Starting: Running benchmark(s). Stand by!

•

Method                        Mean        Samples    Change
-----------------------------------------------------------
pg_query_parse_protobuf    5.03 ms    1,934/1,960    -0.71%

Flamegraphs

pg_query_parse():

pg_query_parse_protobuf():

[duckinator]

The initial flamegraph for pg_query_parse_protobuf() was wrong. Regenerating it fixed the problem, and I uploaded a new one + updated my summary accordingly.

[lfittl]

It might be interesting to include the deserialization in Rust as well, so we can get a sense for where to optimize (if we were to optimize the serialization itself).

For the protobuf parse benchmark we can use the same mechanism the crate currently uses. For JSON we could (just for testing) use the mechanism that pg_parse uses (which shares a common history with this crate, but we since diverged to focus on the Protobuf format).

[vrmiguel]

Probably out of scope of this PR but a cool addition would be comparisons to datafusion-sqlparser-rs

[seanlinsley]

Crates should be ordered alphabetically. Also it's better to use 0.8 (now 0.10) when you don't intend to pin to a patch version.

[seanlinsley]

Since the actual code in the benchmarks is so simple, I wonder if they should be merged into a single benchmark file:

use brunch::Bench;
use pg_query::bindings::*;
use std::ffi::{c_char, CString};

brunch::benches!(
    Bench::new("parse_json").run_seeded_with(seed, |query| unsafe { pg_query_parse(query.as_ptr() as *const c_char) }),
    Bench::new("parse_protobuf").run_seeded_with(seed, |query| unsafe { pg_query_parse_protobuf(query.as_ptr() as *const c_char) }),
    Bench::new("parse_summary").run_seeded_with(seed, |query| unsafe { pg_query_parse_summary(query.as_ptr() as *const c_char, 0, 0) }),
);

fn seed() -> CString {
    CString::new(build_query(100)).unwrap()
}

fn build_query(table_references: i32) -> String {
    let mut query = "SELECT * FROM t".to_string();
    for i in 0..table_references {
        query = format!("{query} JOIN t{i} ON t.id = t{i}.t_id AND t{i}.k IN (1, 2, 3, 4) AND t{i}.f IN (SELECT o FROM p WHERE q = 'foo')");
    }
    query
}