Hello there!

I'm writing a library (roughly) à la Google's ProtoBuf that, in contrast to ProtoBuf, supports streaming parts of the message.

I've made great progress in implementing small Encoder, Decoder, Encodable and Decodable traits, but there's one giant issue where I'm stuck.

Currently I want to support two views that can be streamed:

• A Blob for raw bytes
• A Readable<T> for a sequence of nested messages T

We'll focus on the Blob because the issue is identical.

pub struct Blob<'a> {
    size: u64,
    stream: Arc<Mutex<dyn futures::stream::Stream<Item = std::result::Result<bytes::BytesMut, std::io::Error> + 'a>>,
}

In Encoder, the consumer can pipe e.g. a file into a stream, create a Message or something similar with a Blob that wraps that stream and turn over that object to my encoder, which will consume the Blob. This works wonderfully.

In Decoder, I need to return 'views' on the underlying AsyncRead. Currently, I return futures::stream::Stream<Item = std::result::Result<bytes::BytesMut, std::io::Error>.

// Example message
// #[derive(Encode, Decode)] // in a later stage
pub struct AddFile<'a> {
    path: String,
    content: Blob<'a>,
}

This is where I construct my Blob when reading a message:

impl <R: AsyncRead + Unpin> Decoder for AsyncReadDecoder<R> {

    async fn decode_blob(&mut self) -> Result<Blob<'_>> {
        let size = self.decode_u64().await?;
        let reader: FramedRead<&mut R, BytesCodec> = FramedRead::new(&mut self.reader, BytesCodec::new());
        Ok(Blob {
            size,
            stream: Arc::new(Mutex::new(reader)),
        })
    }

}

Testing the decoder on AddFile:

impl <'a> Decodable for AddFile<'a> {
    async fn decode<D: Decoder>(decoder: &mut D) -> Result<Self> {
        Ok(AddFile {
            path: decoder.decode().await?,
            content: decoder.decode_blob().await?,
        })
    }
}

I get this lifetime related error:

error: lifetime may not live long enough
   --> src/lib.rs:332:13
    |
330 |       impl <'a> Decodable for AddFile<'a> {
    |             -- lifetime `'a` defined here
331 |           async fn decode<D: Decoder>(decoder: &mut D) -> Result<Self> {
    |                                                - let's call the lifetime of this reference `'1`
332 | /             Ok(AddFile {
333 | |                 path: decoder.decode().await?,
334 | |                 content: decoder.decode_blob().await?,
335 | |             })
    | |______________^ associated function was supposed to return data with lifetime `'a` but it is returning data with lifetime `'1`

I think I get the issue: the reason this doesn't work is because returning a view of a &mut AsyncRead would give two mutable references to the same &mut AsyncRead. My question is, then: how do you solve this? Is there some kind of pattern I'm missing? Or is this genuinely difficult to achieve in Rust?

Thanks,
Sam

1 post - 1 participant

Read full topic