roto/TASKS.md

Tasks for Tonic Integration

This document outlines the steps required to integrate the roto protobuf library with the tonic gRPC framework.

Goals

• Provide a tonic::codec::Codec implementation for roto messages.
• Enable zero-allocation decoding of gRPC requests and responses.
• Support efficient encoding of gRPC messages using roto's Builder pattern.
• Generate tonic-compatible service traits and client/server boilerplate via protoc-gen-roto.

1. Runtime Changes (roto_runtime)

• Add bytes as a dependency to roto_runtime.
• Modify ProtoBuilder to support writing to bytes::BufMut or provide a specialized BufMut based builder to facilitate integration with tonic::codec::EncodeBuf.
• Design and implement "Owned" message support:
  • Create a mechanism (likely via codegen) to generate structs that hold bytes::Bytes and the field offsets (e.g., OwnedMyMessage).
  • These structs will serve as the owned types required by tonic's Codec.
  • Add a method to convert an OwnedMyMessage into a zero-allocation Reader (e.g., reader(&self) -> MyMessage<'_>).

2. Tonic Codec Implementation

• Implement tonic::codec::Codec for roto messages.
• Implement tonic::codec::Decoder for roto messages:
  • The decoder should take a DecodeBuf and produce an OwnedMyMessage.
  • It must perform the initial scan of the buffer to populate the field offsets.
• Implement tonic::codec::Encoder for roto messages:
  • The encoder should take an OwnedMyMessage and write its internal bytes::Bytes to the EncodeBuf.
  • Since roto responses are built using Builder directly into a buffer, the Encoder will primarily handle copying these pre-built buffers.

3. Code Generation Extensions (protoc-gen-roto)

• Update the generator to produce OwnedMyMessage structs in addition to the Reader and Builder structs.
• Generate the OwnedMyMessage::new(data: bytes::Bytes) method for initial decoding.
• Implement gRPC service generation:
  • Generate tonic-compatible service traits (using #[tonic::async_trait]).
  • Generate client and server boilerplate that uses OwnedMyMessage as the request and response types.
  • Ensure the generated code correctly maps protobuf services to Rust traits.

4. Testing and Validation

• Create a test gRPC project using the updated protoc-gen-roto.
• Implement a sample gRPC service with:
  • A unary call.
  • A server-streaming call.
  • A client-streaming call.
  • A bidirectional-streaming call.
• Verify that the integration is zero-allocation on the reading path.
• Perform benchmark tests to compare performance with prost.