Cannot encode message google.protobuf.Any in the v2.0.0-alpha

[biziosan]

Describe the bug

When I use google.Any in my RPC arguments I get a "[internal] cannot encode message google.protobuf.Any to JSON: "type.googleapis.com/TestObject" is not in the type registry" message when calling the RPC.

To Reproduce

D:\projects\sd\versioned\sarah\projects\terra\node_modules\@connectrpc\connect\dist\cjs\protocol-connect\error-json.js:57
    const error = new connect_error_js_1.ConnectError(message !== null && message !== void 0 ? message : "", code, metadata);
                  ^

ConnectError: [internal] cannot encode message google.protobuf.Any to JSON: "type.googleapis.com/TestObject" is not in the type registry
    at errorFromJson (D:\projects\sd\versioned\sarah\projects\terra\node_modules\@connectrpc\connect\dist\cjs\protocol-connect\error-json.js:57:19)
    at next (D:\projects\sd\versioned\sarah\projects\terra\node_modules\@connectrpc\connect-web\dist\cjs\connect-transport.js:90:68)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.unary (D:\projects\sd\versioned\sarah\projects\terra\node_modules\@connectrpc\connect-web\dist\cjs\connect-transport.js:55:20)
    at async Object.requestTest (D:\projects\sd\versioned\sarah\projects\terra\node_modules\@connectrpc\connect\dist\cjs\promise-client.js:70:26) {
  rawMessage: 'cannot encode message google.protobuf.Any to JSON: "type.googleapis.com/TestObject" is not in the type registry',
  code: 13,
  metadata: Headers {
    [Symbol(headers list)]: HeadersList {
      cookies: null,
      [Symbol(headers map)]: Map(5) {
        'connection' => { name: 'connection', value: 'keep-alive' },
        'content-length' => { name: 'content-length', value: '145' },
        'content-type' => { name: 'content-type', value: 'application/json' },
        'date' => { name: 'date', value: 'Wed, 28 Aug 2024 10:54:38 GMT' },
        'keep-alive' => { name: 'keep-alive', value: 'timeout=5' }
      },
      [Symbol(headers map sorted)]: null
    },
    [Symbol(guard)]: 'none'
  },
  details: [],
  cause: undefined
}

Node.js v20.14.0

Process finished with exit code 1

Returning a non-Any message works as expected.

---

These are the .proto files used:

import "google/protobuf/any.proto";

message TestAny
{
    google.protobuf.Any object = 1000;
}

message TestObject
{
    uint32 n = 1000;
}

message TestRequest
{
    string r = 1000;
}

service TestService
{
    rpc RequestTest(TestRequest) returns (TestAny);
}

This reduced example generates the error:

function createServer()
{
    const rpcHandler = cnrpc.connectNodeAdapter(
        {
            routes: function(router: crpc.ConnectRouter) {
                router.service(TestService, {
                    requestTest(request: TestRequest): TestAny
                    {
                        const o = proto.create(TestObjectSchema, { n: 9 });
                        const a = proto_wkt.anyPack(TestObjectSchema, o);

                        return proto.create(TestAnySchema, { object: a });
                    },
                });
            }
        }
    );

    http.createServer(rpcHandler).listen(8099);
}

(async () => {
    createServer();

    const transport = crpc_web.createConnectTransport({ baseUrl: 'http://127.0.0.1:8099' });

    const client = crpc.createPromiseClient(TestService, transport);
    client.requestTest({ r: 'test' }).then((response) => {
            console.log(response);
    });
})();

Environment (please complete the following information):

• @connectrpc/connect-web version: 2.0.0-alpha.1
• @connectrpc/connect-node version: 2.0.0-alpha.1
• Frontend framework and version: n/a
• Node.js version: v20.14.0
• Browser and version: n/a

Additional context
Add any other context about the problem here.

[timostamm]

Hey @biziosan, thanks for giving the alpha a try!

google.protobuf.Any stores arbitrary messages in it's fields string type_url (for the type name) and bytes value (for the serialized content).

In the JSON format, Any doesn't simply serialize the two fields: Instead, the contained message is serialized to JSON, and the type name is added in the @type property.

So to serialize Any to JSON, it's necessary to parse the contained message first. And for this step, it's necessary to know the schema. Similar for parsing Any from JSON.

There is a simple solution to the problem: You can create a registry with the types you want to allow for Any, and pass the registry to the router via jsonOptions:

import { createRegistry } from "@bufbuild/protobuf";
import { connectNodeAdapter } from "@connectrpc/connect";

connectNodeAdapter({
  routes,
  jsonOptions: {
    registry: createRegistry(TestObjectSchema),
  },
});

[biziosan]

Hi @timostamm, thank you for this new version! I am looking forward to working with it.

Thank you for looking at this issue. I made the changes you suggested (the new code is below), and it works now! I added the registry for both the encode and decode sides. In my main application, I already have the full registry, so I will give it to the JSON options.

function createServer()
{
    const rpcHandler = cnrpc.connectNodeAdapter(
        {
            routes: function(router: crpc.ConnectRouter) {
                router.service(TestService, {
                    requestTest(request: TestRequest): TestAny
                    {
                        const o = proto.create(TestObjectSchema, { n: 9 });
                        const a = proto_wkt.anyPack(TestObjectSchema, o);

                        return proto.create(TestAnySchema, { object: a });
                    },
                });
            },
            jsonOptions: {
                registry: proto.createRegistry(TestObjectSchema),
            }
        }
    );

    http.createServer(rpcHandler).listen(8099);
}

(async () => {
    createServer();

    const transport = crpc_web.createConnectTransport(
        {
            baseUrl: 'http://127.0.0.1:8099',
            jsonOptions: {
                registry: proto.createRegistry(TestObjectSchema),
            }
        });

    const client = crpc.createPromiseClient(TestService, transport);
    client.requestTest({ r: 'test' }).then((response) => {
            console.log(response);
    });
})();