Unable to use Google well known types

[respinha]

protobuf.js version: 6.8.6

When I attempt to encode a message from a Google well known type (like google.protobuf.Struct) I get an empty field in the decoded message IF my path includes google/protobuf.
Is this a bug or do we need to provide the path in some other way. I assume the problem starts when loading the commons wrapper here.

const protobuf = require('protobufjs');

let root = new protobuf.Root();
const protoFilePath = '<path to a file which imports google.protobuf.Struct>';
const protoRoot = .............;
root.resolvePath = function (origin, target) {
  // ignore the same file
  if (target == protoFilePath) {
    return protoFilePath;
  }
  // Resolved target path for the import files
  return protoRoot + target;
};
const struct = {
  fields: {
    label: {
      struct_value: {
        fields: {
          another_label: {
            number_value: 10
          }
        }
      }
    }
  }
};
root = root.loadSync(protoFilePath);
const MyType = root.lookupType('MyType');
const msg = MyType.create({
  struct_message: struct
});

const buffer = MyType.encode(msg).finish();

console.log('Decoded message', MyType.decode(buffer));

Decoded message: Struct { fields: { label: Value {} } } }

Note: this was working with protobuf.js version 5.

[respinha]

Bump: @dcodeIO, is this a bug or is there another way of using these types?

[dcodeIO]

This might be due to protobuf.js implicitly converting field names from underscore_notation to camelCaseNotation since v6 (like google's js reference js impl does). You can try

root.loadSync(protoFilePath, { keepCase: true })

to retain underscore notation or change the struct contents accordingly.

[respinha]

I loaded with { keepCase: true } and still the same problem.

[dcodeIO]

What happens if you do:

const struct = {
  fields: {
    label: {
      kind: "struct_value",
      struct_value: {
        fields: {
          another_label: {
            kind: "number_value",
            number_value: 10
          }
        }
      }
    }
  }
};

[respinha]

Still the same result...

[dcodeIO]

Just edited this to use the string field names, can you try again? I believe it looks for the virtual oneof field there to determine which type of value is provided.

[respinha]

Tried and still this
Struct { fields: { label: Value {} } } }

[dcodeIO]

Hmm, what happens if you also do

const msg = MyType.fromObject({
  struct_message: struct
});

If that also doesn't work, can you provide the .proto definition for MyType?

[respinha]

I am using these protos: https://github.com/restorecommerce/protos. The proto definition is in io/restorecommerce/resource_base.proto (ReadRequest). It has a filter field which is of type google.protobuf.Struct. It has always worked with grpc-node which fallsback to protobufjs v5 when loading proto definitions but not if we specifically try v6.

[dcodeIO]

Thanks. Seems that there is an issue with keepCase in conjunction with the struct type, because src/common.js explicitly defines it in camelCase notation, which isn't converted back when keepCase = true.

This appears to work:

var root = new protobuf.Root();
root.load("google/protobuf/struct.proto", function() {
  protobuf.parse(`syntax = "proto3"; message MyType { google.protobuf.Struct filter = 1; }`, root);
  var MyType = root.MyType;
  var myType = MyType.fromObject({
    filter: {
      fields: {
        number: {
          numberValue: 10
        },
        struct: {
          structValue: {
            fields: {
              number: {
                numberValue: 20
              }
            }
          }
        }
      }
     }
  });
  console.log(myType);
});

[respinha]

Oh, interesting! Then is it a bug when passing the options to common.js?

[dcodeIO]

Yes, this is a bug affecting keepCase. A proper fix could be to move field renaming out of the parser into construction of the reflection structure, in turn defining all of common.js in underscore notation.

[respinha]

Btw, what is the use of those custom wrappers for Google well known types? Wouldn't they be parsed correctly in the "standard" way?