GTS types serialize to JSON only. We want to send GTS entities as MessagePack and over gRPC. This can be added later on top of gts-macros without touching the JSON path.
Current state
#[struct_to_gts_schema] adds Serialize/Deserialize derives to base structs. Nested structs (base = Parent) can't implement serde traits directly. The macro generates GtsSerialize/GtsDeserialize for them instead (gts-macros/src/lib.rs, traits in gts/src/schema.rs).
- Both traits are generic over
serde::Serializer/serde::Deserializer. Nothing in the type signatures ties them to serde_json.
- The generated
gts_serialize calls serialize_struct. The generated gts_deserialize calls deserialize_struct and implements visit_map only, no visit_seq.
- A nested type is serialized as an object under the base struct's generic field (
payload). The macro doesn't use #[serde(flatten)].
- Entity I/O in
gts uses serde_json, plus serde-saphyr for YAML input in files_reader.rs. There are no protobuf or MessagePack dependencies.
- Schemas are JSON Schema draft-07.
Using gts-rust with gRPC today
No changes to gts-rust are needed. Carry the JSON as bytes in a proto field:
message GtsEvent {
string type_id = 1;
bytes payload = 2; // JSON produced by serde_json
}
The sender serializes the full base struct, for example BaseEventV1<AuditPayloadV1<D>>, with serde_json::to_vec. A nested type can't be serialized on its own, so always send the base struct. type_id repeats the type field from the JSON so the receiver can route the message without parsing the payload.
A receiver that knows the Rust type calls serde_json::from_slice into it. A receiver that doesn't parses the payload into serde_json::Value and checks it with GtsStore::validate_payload(type_id, &value). The type's schemas must be registered in the store first.
This is the envelope option from Stage 2, done in application code.
Stage 1: MessagePack
MessagePack is a serde format, so the existing impls should work through rmp-serde without macro changes. This hasn't been tested.
One known problem: rmp-serde encodes structs as arrays by default. The generated deserializer has no visit_seq, so nested types won't decode in that mode. Either require map mode (Serializer::with_struct_map()) or generate visit_seq as well. Map mode keeps field names on the wire and matches the JSON shape. Array mode is smaller but depends on field order.
Work:
- optional
msgpack feature in gts with encode/decode helpers
- round-trip tests over the existing inheritance fixtures: base, nested, generic payload, unit structs
- pick map mode only, or add
visit_seq
Stage 2: Protobuf / gRPC
Protobuf isn't a serde format. prost has its own Message trait and identifies fields by tag number, so the serde impls can't be reused. Two options.
Envelope. A fixed message with the serialized payload inside:
message GtsEntity {
string type_id = 1;
bytes payload = 2; // JSON or MessagePack
}
No macro changes. Payloads stay untyped on the wire, and clients can't use generated types for them.
Generated messages. A new macro attribute or derive emits a .proto message per GTS struct and From/TryFrom conversions to the prost types. gts-macros-cli would write the .proto files the same way it writes .schema.json today. Because nested types already serialize as a separate object under the generic field, each struct maps to its own message and the generic field maps to a message field.
Open questions for generated messages:
- How to represent a generic field (
payload: P): one message per concrete instantiation, a oneof over known child types, or google.protobuf.Any.
- Where tag numbers come from. Deriving them from field order breaks when fields are reordered. An explicit per-field attribute is stable but adds annotations to every GTS struct.
- How tag numbers stay stable across minor versions of a type.
- Whether
schema_evolution.rs should also check .proto compatibility, or whether JSON Schema compatibility is enough.
If a gRPC consumer needs this before generated messages are designed, ship the envelope first. It doesn't block the second option.
Out of scope
- Changes to the JSON format or JSON Schema output.
GTS types serialize to JSON only. We want to send GTS entities as MessagePack and over gRPC. This can be added later on top of
gts-macroswithout touching the JSON path.Current state
#[struct_to_gts_schema]addsSerialize/Deserializederives to base structs. Nested structs (base = Parent) can't implement serde traits directly. The macro generatesGtsSerialize/GtsDeserializefor them instead (gts-macros/src/lib.rs, traits ingts/src/schema.rs).serde::Serializer/serde::Deserializer. Nothing in the type signatures ties them toserde_json.gts_serializecallsserialize_struct. The generatedgts_deserializecallsdeserialize_structand implementsvisit_maponly, novisit_seq.payload). The macro doesn't use#[serde(flatten)].gtsusesserde_json, plusserde-saphyrfor YAML input infiles_reader.rs. There are no protobuf or MessagePack dependencies.Using gts-rust with gRPC today
No changes to gts-rust are needed. Carry the JSON as bytes in a proto field:
The sender serializes the full base struct, for example
BaseEventV1<AuditPayloadV1<D>>, withserde_json::to_vec. A nested type can't be serialized on its own, so always send the base struct.type_idrepeats thetypefield from the JSON so the receiver can route the message without parsing the payload.A receiver that knows the Rust type calls
serde_json::from_sliceinto it. A receiver that doesn't parses the payload intoserde_json::Valueand checks it withGtsStore::validate_payload(type_id, &value). The type's schemas must be registered in the store first.This is the envelope option from Stage 2, done in application code.
Stage 1: MessagePack
MessagePack is a serde format, so the existing impls should work through
rmp-serdewithout macro changes. This hasn't been tested.One known problem:
rmp-serdeencodes structs as arrays by default. The generated deserializer has novisit_seq, so nested types won't decode in that mode. Either require map mode (Serializer::with_struct_map()) or generatevisit_seqas well. Map mode keeps field names on the wire and matches the JSON shape. Array mode is smaller but depends on field order.Work:
msgpackfeature ingtswith encode/decode helpersvisit_seqStage 2: Protobuf / gRPC
Protobuf isn't a serde format. prost has its own
Messagetrait and identifies fields by tag number, so the serde impls can't be reused. Two options.Envelope. A fixed message with the serialized payload inside:
No macro changes. Payloads stay untyped on the wire, and clients can't use generated types for them.
Generated messages. A new macro attribute or derive emits a
.protomessage per GTS struct andFrom/TryFromconversions to the prost types.gts-macros-cliwould write the.protofiles the same way it writes.schema.jsontoday. Because nested types already serialize as a separate object under the generic field, each struct maps to its own message and the generic field maps to a message field.Open questions for generated messages:
payload: P): one message per concrete instantiation, aoneofover known child types, orgoogle.protobuf.Any.schema_evolution.rsshould also check.protocompatibility, or whether JSON Schema compatibility is enough.If a gRPC consumer needs this before generated messages are designed, ship the envelope first. It doesn't block the second option.
Out of scope