[BigQuery] How to instantiate request

[FEC-bendingspoons]

Hello!

I am trying to use the insert_all method:

google-apis-rs/gen/bigquery2/src/api.rs

Line 4841 in beb110a

pub fn insert_all(&self, request: TableDataInsertAllRequest, project_id: &str, dataset_id: &str, table_id: &str) -> TabledataInsertAllCall<'a> {

This requires me to instantiate a TableDataInsertAllRequest struct:

google-apis-rs/gen/bigquery2/src/api.rs

Line 3093 in beb110a

pub struct TableDataInsertAllRequest {

This requires me to instantiate a vector of TableDataInsertAllRequestRows:

google-apis-rs/gen/bigquery2/src/api.rs

Line 3853 in beb110a

pub struct TableDataInsertAllRequestRows {

This require me to provide a JsonObject for each row:

google-apis-rs/gen/bigquery2/src/api.rs

Line 2035 in beb110a

pub struct JsonObject(Option<HashMap<String, JsonValue>>);

But it has private members and no initializer that I can see. How can I create it? What am I missing?

[Byron]

Thanks for letting me know in such an easy to follow fashion! Apologies for turning this legitimate issue into a discussion, #328 is the re-created issue from it to which you might want to subscribe for updates.

That said, to me it appears that the tuple-structs the system generates are generally lacking the pub specifier and I couldn't think of any other reason than oversight.

My suggestion is to vendor the code, make the needed adjustments, and see if the valid request can be made like that. I have made the necessary adjustment to the generator and regenerated the code so you can take it from main. Please let me know in the issue if this works for you, and I could make a new release for bigquery2 specifically. Thanks.

[FEC-bendingspoons]

As always, it's a pleasure interacting with you.

I can confirm that the new version currently on main (rev: 77f097d) works 🚀 .

It is a bit unwieldy however. I have something like

#[derive(Serialize)]
struct Event {
 // ... various fields
}

RIght now I am keeping a buffer of them ready to be sent

let mut buffer: Vec<JsonObject> = Vec::with_capacity(CAPACITY);

loop {
  match channel.recv() {
    Some(event) => {
       // here I have to do some work
      let values: HashMap<String, JsonValue> = serde_json::to_value(event)
          .unwrap()
          .as_object()
          .unwrap()
          .into_iter()
          .map(|(key, value)| (key.clone(), JsonValue(value.clone())))
          .collect();

      buffer.push(JsonObject(Some(values)));
    }
    _ => {}
  }
  // ....
}

I don't have many ideas on how to make it better :(

[FEC-bendingspoons]

I played around a little bit and I think that it should be possible to add something like:

impl<T> TryFrom<T> for JsonObject where T: Serialize {
    type Error = serde_json::Error;

    fn try_from(value: T) -> Result<Self, Self::Error> {
        let values_map: HashMap<String, JsonValue> = serde_json::to_value(value)?
            .as_object()
            .ok_or_else(|| {
                serde_json::Error::custom("value needs to be representable as a JSON object")
            })?
            .into_iter()
            .map(|(key, value)| (key.clone(), JsonValue(value.clone())))
            .collect();

        Ok(JsonObject(Some(values_map)))
    }
}

but the compiler complains about a duplicated conformance, and I cannot figure out why 😢

[Byron]

Thank you :). And you are welcome.

And even if the above would work, the problem is that JsonObject is fully generated and it would be difficult/complicated to add implementations for it.

Since it's now working I would take this home as a win and hope that with utility functions it should be possible to ease the pain of using it.

[FEC-bendingspoons]

Yea, I share the same feeling. Thanks for the support!

[FEC-bendingspoons]

@Byron

I think something went wrong somewhere with the release... I can see that the regeneration commit (77f097d) bumped the version to 3.0.1 6 days ago, but according to crates.io version 3.0.1 was released 25 days ago.

Am I missing something, or are things really broken?

[Byron]

Sorry for the hassle, this is quite confusion indeed and I don't know what happened there. So I just fixed it by releasing v3.0.2, which should be the version you are looking for.

[FEC-bendingspoons]

Amazing, thank you!