So this post is partially because I am trying to understand the problem better, part of it is me wondering if there is any better solution that I'm not aware of. First let me add a bit of context.

I'm writing a parser for a file format that contains "links", let's call it the foo format. (I'm simplifying a bit in an effort to keep the terminology as clear as possible without having to go into detail too much) The parsing bit is already written, now I just have to pump the data into it correctly. To do this I am writing the following struct and impl:

pub struct FooReader<R: std::io::Read> {
    inner: ...., // more on this in a sec
}

impl<R: std::io::Reader> FooReader<R>{
    // taking ownership of reader
    pub fn from_reader(reader: R) -> FooReder<R> {
    ...
    }
}

and I implement Iterator for it

impl<R: std::io::Read> Iterator for FooReader<R> {
    type Item = Result<Link, FooReaderError>;

    fn next(&mut self) -> Option<Self::Item> {
        match self.inner.next() {
            ... // again more on this bit in a sec.
    }
}

One quirk of the format (which I don't control btw) is that the first 4 lines of a file are ascii header lines with some metadata that I can parse, and after that the rest of the file is zlib compressed data.

I can read the header lines by way of BufReader::new(reader).lines() and after that I can read body lines by using BufReader::new(ZlibDecoder::new(buffered_header_reader.into_inner())).lines();

(with reader being the original raw reader we took ownership of in from_reader)

The difficulty comes from the fact that I have to read "different types" from the same Read stream. Obviously I'd like to just give my users an instance of FooReader that can just read the header/body as necessary similar to what the csv crate does.

My original idea was to use an enum:

pub enum InnerFooReader<R: std::io::Read> {
   Header(BufReader(R)),
   Body(Lines<BufReader<ZlibDecoder<R>>>)
}

with the idea to do something along the lines of:

 fn next(&mut self) -> Option<Self::Item> {
        self.inner = match self.inner {
             Header(r) => {
            // read header from r, and store it in 
            // self (colning where necessary to take ownership)
            
            // now wrap r in Lines<BufReader<ZlibDecoder<R>>> 
            // and return the other enum variant
              },
          Body(r) => Body(r),

    };
       self.inner.next()
}

The idea being that reading from the iterator always returns a Link, but if we're still in the header part we just read that, store the info away fro later in necessary, and then read from the body as normal.

The borrow checker does not like this because technically we don't have ownership of self.inner because we are behind a &mut self even though we do own the underlying data.

I have actually found and read this blog post which talks about a similar issue and recommend wrapping the stream in an Option so I can "temporarily take ownership using Option::take" which is probably doable but is a little unsatisfying ergonomically.

Before I found this option I also worked on a solution that simply reads the header part from the reader immediately during from_reader stores the header part and then permanently wraps the reader in ZlibDecoder like above so we can just read from it. This works fine, but it does read from the reader upon construction. Since we take ownership of the std::io::Read anyway I guess this isn't a huge deal, but again feels unsatisfactory from an abstraction point of view.

So to sum up, my actual questions are:

1. While certainly solid the blog post dates from 2018 and so I'm wondering if any new developments since then have enabled a more elegant solution
2. are there any design patterns I overlooked for this?
3. Do you have any other feedback about this setup?

Thanks a lot!

4 posts - 3 participants

Read full topic