Record findings

TASKS.md

@@ -1,44 +1,123 @@
-# Tasks for Tonic Integration
+# Task: Verify Generated Code Works in `no_std` / `no_alloc`
 
-This document outlines the steps required to integrate the `roto` protobuf library with the `tonic` gRPC framework.
+**Goal:** Ensure all generated protobuf code compiles and runs in `no_std` + `no_alloc` environments. Features requiring `alloc` must be gated behind feature flags in the proto generator's output.
 
-## Goals
-- Provide a `tonic::codec::Codec` implementation for `roto` messages.
-- Enable zero-allocation decoding of gRPC requests and responses.
-- Support efficient encoding of gRPC messages using `roto`'s `Builder` pattern.
-- Generate `tonic`-compatible service traits and client/server boilerplate via `protoc-gen-roto`.
+---
 
-## 1. Runtime Changes (`roto_runtime`)
-- [x] Add `bytes` as a dependency to `roto_runtime`.
-- [x] Modify `ProtoBuilder` to support writing to `bytes::BufMut` or provide a specialized `BufMut` based builder to facilitate integration with `tonic::codec::EncodeBuf`.
-- [x] Design and implement "Owned" message support:
-    - [x] Create a mechanism (likely via codegen) to generate structs that hold `bytes::Bytes` and the field offsets (e.g., `OwnedMyMessage`).
-    - [x] These structs will serve as the owned types required by `tonic`'s `Codec`.
-    - [x] Add a method to convert an `OwnedMyMessage` into a zero-allocation `Reader` (e.g., `reader(&self) -> MyMessage<'_>`).
+## Findings
 
-## 2. Tonic Codec Implementation
-- [x] Implement `tonic::codec::Codec` for `roto` messages.
-- [x] Implement `tonic::codec::Decoder` for `roto` messages:
-    - [x] The decoder should take a `DecodeBuf` and produce an `OwnedMyMessage`.
-    - [x] It must perform the initial scan of the buffer to populate the field offsets.
-- [x] Implement `tonic::codec::Encoder` for `roto` messages:
-    - [x] The encoder should take an `OwnedMyMessage` and write its internal `bytes::Bytes` to the `EncodeBuf`.
-    - [x] Since `roto` responses are built using `Builder` directly into a buffer, the `Encoder` will primarily handle copying these pre-built buffers.
+**Already clean (no_std + no_alloc):**
+- All reader structs (`Message<'a>`) — zero-copy accessors over `&[u8]`
+- All field accessors, enums, builders (`MessageBuilder<'b>`), typed iterators
+- Core runtime: `ProtoAccessor`, `ProtoBuilder`, `RevBuilder`, `read_varint`, `write_varint`, all iterators
 
-## 3. Code Generation Extensions (`protoc-gen-roto`)
-- [x] Update the generator to produce `OwnedMyMessage` structs in addition to the `Reader` and `Builder` structs.
-- [x] Generate the `OwnedMyMessage::new(data: bytes::Bytes)` method for initial decoding.
-- [x] Implement gRPC service generation:
-    - [x] Generate `tonic`-compatible service traits (using `#[tonic::async_trait]`).
-    - [x] Generate client and server boilerplate that uses `OwnedMyMessage` as the request and response types.
-    - [x] Ensure the generated code correctly maps protobuf services to Rust traits.
+**Needs `alloc` feature gate:**
+- `RotoMessage` trait — uses `bytes::Bytes` (has `Arc` internally)
+- `RotoOwned` trait — same issue
+- `OwnedMessage` structs generated by codegen
+- `RevBuilder::with()` — uses `Vec<_>` to collect raw fields
+- `Bytes`, `BytesMut` imports in `DATA_IMPORTS`
 
-## 4. Testing and Validation
-- [x] Create a test gRPC project using the updated `protoc-gen-roto`.
-- [ ] Implement a sample gRPC service with:
-    - [ ] A unary call.
-    - [ ] A server-streaming call.
-    - [ ] A client-streaming call.
-    - [ ] A bidirectional-streaming call.
-- [ ] Verify that the integration is zero-allocation on the reading path.
-- [ ] Perform benchmark tests to compare performance with `prost`.
+**Needs `std` feature gate:**
+- All service code (`SERVICE_IMPORTS`) — uses `std::sync::Arc`, `std::pin::Pin`, `String`, etc.
+- `std::error::Error for RotoError` (already gated)
+
+---
+
+## Plan
+
+### Phase 1: Runtime (`runtime/src/lib.rs`, `runtime/Cargo.toml`)
+
+1. **Gate `RotoMessage` and `RotoOwned`** behind `#[cfg(feature = "alloc")]`
+2. **Gate `BufMutBuilder`** behind `#[cfg(feature = "alloc")]` — only useful with `BytesMut`, which requires alloc
+3. **Gate `use bytes::BufMut`** — only import it conditionally
+4. **Wire up `bytes` features in Cargo.toml:**
+   ```toml
+   bytes = { version = "1.7", default-features = false }
+   # ...
+   [features]
+   default = ["std", "alloc"]
+   std = ["bytes/std"]
+   alloc = ["bytes"]
+   ```
+
+### Phase 2: Codegen — `DATA_IMPORTS` & Generated Data Code (`codegen/src/generator/mod.rs`)
+
+5. **Split `DATA_IMPORTS`** into unconditional + gated sections:
+   ```rust
+   // Always emitted
+   use roto_runtime::{ProtoAccessor, ProtoBuilder, RevBuilder, Result, RotoError, read_varint, RepeatedFieldIterator};
+   use core::str;
+   use bytes::{Buf, BufMut};
+
+   // Only emitted when alloc feature is enabled
+   #[cfg(feature = "alloc")]
+   use bytes::{Bytes, BytesMut};
+   #[cfg(feature = "alloc")]
+   use roto_runtime::RotoMessage;
+   ```
+
+6. **Gate `OwnedMessage` structs + `RotoMessage` impls** with `#[cfg(feature = "alloc")]`
+
+7. **Fix `RevBuilder::with()`** — replace `let fields: Vec<_> = ... .collect()` with a forward iteration that doesn't allocate. The reverse order doesn't matter for protobuf encoding (fields are independent):
+   ```rust
+   // Before: let fields: Vec<_> = msg.accessor.raw_fields().collect();
+   //         for (raw, _) in fields.iter().rev() { ... }
+   // After:  for (raw, _) in msg.accessor.raw_fields() { ... }
+   ```
+   Alternatively, gate the `with()` method itself behind `#[cfg(feature = "alloc")]`.
+
+### Phase 3: Codegen — Service Code (`codegen/src/generator/mod.rs`)
+
+8. **Gate `SERVICE_IMPORTS`** with `#[cfg(feature = "std")]`
+9. **Gate all service traits, server impls, and `*Server` structs** with `#[cfg(feature = "std")]`
+
+### Phase 4: Generated Consumer Setup
+
+10. **Emit a `Cargo.toml`** (or instruct consumers) with the right feature setup:
+    ```toml
+    [features]
+    default = ["std"]
+    std = ["roto-runtime/std", "alloc"]
+    alloc = ["roto-runtime/alloc", "bytes"]
+    ```
+
+### Phase 5: Validation
+
+11. **Update `examples/no_std_test`** to build with `--no-default-features` and verify compilation
+12. **Run existing tests** with default features to ensure nothing breaks
+13. **Add a compile-test** that verifies `--no-default-features` builds succeed for generated code
+
+---
+
+## Dependency Graph
+
+```
+no_std + no_alloc  →  readers, enums, builders, iterators
+         ↓ alloc   →  + OwnedMessage, RotoMessage trait
+         ↓ std     →  + service code, std::error::Error
+```
+
+## Risk Assessment
+
+| Risk | Mitigation |
+|------|-----------|
+| Breaking existing consumers who use `OwnedMessage` | `OwnedMessage` was already behind a manual `#[cfg]` in the `no_std_test` example — consumers aren't relying on it unconditionally |
+| `BufMutBuilder` removal in no-alloc mode | It's only useful with `BytesMut` anyway, which requires alloc |
+| `RevBuilder::with()` change | Forward iteration is semantically equivalent for protobuf encoding |
+
+## Files to Modify
+
+| File | Changes |
+|------|---------|
+| `runtime/src/lib.rs` | Gate `RotoMessage`, `RotoOwned`, `BufMutBuilder`, `use bytes::BufMut` |
+| `runtime/Cargo.toml` | Wire `alloc` → `bytes` dependency, add `bytes/std` to `std` feature |
+| `codegen/src/generator/mod.rs` | Split `DATA_IMPORTS`, gate `OwnedMessage`, gate service code, fix `RevBuilder::with()` |
+| `examples/no_std_test/` | Update to use new feature structure |
+
+## Execution Order
+
+1. Phase 1 (runtime) — foundation
+2. Phase 2 (codegen data imports) — depends on runtime
+3. Phase 3 (codegen services) — independent
+4. Phase 4–5 (consumer setup + validation) — depends on all above