Hello.
I am trying to use serde to implement a serialization format to serialize complex and nested data structures, with the requirement that there are some objects that must be serialized in a specific way when serialized in this format, ie. with a different serialization algorithm than any of the existing 29 types serde supports.
The serialization algorithm for these specific objects does not depend on their type, and the user of the crate must be able to chose a serialization method (and even configure it) per object in the data structure, independently of the type of the object.

The details are unimportant, but let’s say that the crate will need to serialize different objects to different streams (or containers) depending on the user-chosen parameters.
The API is not yet final, but I think the way this will be done is that the serialization crate will provide a number of wrapper objects (Wrapper, WrapperVec, WrapperMap, etc.) that the user will be able to use to contain the objects they want to serialize in a specific way.
For example, struct Foo(i32) would serialize with the default algorithm, but struct Foo(Wrapper(i32)) would serialize the i32 field differently (to a different stream/container).

(This is a simplified example, but the data structures will be deeply nested and may contain wrappers that contain complex structs or enums, that themselves may contain other wrappers, etc. and there will be some advanced and configurable rules to decide where every object is serialized to depending on the nested structure, etc.)

The different wrapper objects (Wrapper, WrapperVec, WrapperMap, etc.) contain either one objet, or a sequence/map, and allow different serialization strategies (and some of them could also contain some configuration options).

I have made some sort of prototype to try to do this with serde, but the problem is that, to be able to do what I descrbed, the Serializer implementation needs to be able to identify the different wrapper objects and act differently than for other objects.
As I didn’t find any way to identify that in the serde API, I used a very ugly hack (please don’t judge me), where I “cheat”: I implement Serialize in my wrappers, but instead of calling the normal methods, I call methods such as serialize_newtype_variant or serialize_tuple_variant, which allows me to provide a type name and variant name that I can then use to identify the type in the Serializer.

First the implementation of Serialize for the wrapper types:

struct Wrapper <T> (T);

impl <T: Serialize> Serialize for Wrapper <T> {
    fn serialize <S> (&self, serializer: S) -> std::result::Result <S::Ok, S::Error>
    where
        S: ser::Serializer,
    {
        serializer.serialize_newtype_variant(
            "Wrapper",
            0,
            "Wrapper",
            &self.0,
        )
    }
}

struct WrapperVec <T> (Vec<T>);

impl <T: Serialize> Serialize for WrapperVec <T> {
    fn serialize <S> (&self, serializer: S) -> std::result::Result <S::Ok, S::Error>
    where
        S: ser::Serializer,
    {
        let mut tuple_variant_serializer = serializer.serialize_tuple_variant(
            "WrapperVec",
            0,
            "WrapperVec",
            self.0.len(),
        )?;
        for el in self.0.iter() {
            ser::SerializeTupleVariant::serialize_field(&mut tuple_variant_serializer, el);
        }
        ser::SerializeTupleVariant::end(tuple_variant_serializer)
    }
}

Then in my Serializer, I can now detect whether it is a real variant or a wrapper by comparing the names:

fn serialize_newtype_variant<T>(
    self,
    name: &'static str,
    variant_index: u32,
    variant: &'static str,
    value: &T,
) -> Result <Self::Ok>
where
    T: ?Sized + Serialize,
{
    match (name, variant_index, variant) {
        ("Wrapper", 0, "Wrapper") => {
            //  Special case/ugly hack: logic to serialize the content of a Wrapper
            ...
        },
        _ => {
            //  Normal case, logic to serialize a real newtype variant
            ...
        },
    }
}

fn serialize_tuple_variant(
    self,
    name: &'static str,
    variant_index: u32,
    variant: &'static str,
    len: usize,
) -> Result<Self::SerializeTupleVariant> {
    match (name, variant_index, variant) {
        ("WrapperVec", 0, "WrapperVec") => {
            //  Special case/ugly hack: logic to serialize a WrapperVec, ie. a sequence that contains multiple values that can be wrapped
            return Ok(TupleVariantSerializer::WrapperVecSerializer {
                ...
            });
        },
        _ => {
            //  Normal case, logic to serialize a real tuple variant
            return Ok(TupleVariantSerializer::RealTupleVariantSerializer {
                ...
            });

        },
    }
}

Now it somehow works in my prototype, and I get the serialization results that I want, but it feels very hacky and not a very robust solution:

• it’s extremely ugly
• semantically catastrophic (uses serde functions for very different things that their “normal” purpose)
• maybe slow because of the string comparison? (though I suppose the compiler could inline that?).
• I don’t know how to pass potential configuration options from the Wrappers to the Serializer
• it is also fragile, as if the user were to use a real enum with name "Wrapper" and a variant named "Wrapper", the serializer would confuse it with the Wrapper object from the crate.
  Now I could use names really unlikely to appear, like "Wrapper1298502Hm%", but it feels like someone mistakenly pasted their password here.
• it would produce ugly/unoptimal/inacceptable serialization with other serialization formats.
  For example, a Wrapper(13i32) would be serialized like a “newtype variant” (ie {"Wrapper": 13} in JSON), where it should be serialized as just 13 (as it really is a newtype struct, not a newtype variant).
  For the time being, I do not plan on supporting any other format, so this is not a blocker, but it could change in the future.

Is there a better/more robust solution with serde?
I would still like to at least be able to serialize all types that implement Serialize (either manually or derived) without requiring a manual implementation of another trait on every type…

1 post - 1 participant

Read full topic