Wednesday, 19 September 2012

Stream.Read returning too Little Data?

I noticed a strange issue I have not noticed anywhere up till now. It seems than when you read from certain types of streams (In this case a stream returned by WebClient.OpenRead) using the Stream.Read method—it can return less data than you asked for, even if there is still data left to read. This is noted in the documentation for the method, but I never noticed because I have never had this problem before.

The problem seems to be solved when using some type of encapsulation(like StreamReader, CryptoStream, GZipStream and the like). It was especially apparent in my case because I had the same code which could, depending on certain conditions, handle compressed data or not. So I had one method that deserializes the data, and another which either passes it the stream it got, or passes a stream that will decompress the data (encapsulating the stream it got). When the decompressing stream was used, it worked fine, but as soon as I tried to read from the stream normally I got some really strange and unpredictable errors (fortunately for me I was checking the result of Stream.Read, and throwing EndOfStreamExceptions when too little data was read).

I suppose the proper way to implement the code would be to use a BinaryReader or the like, but in my case I cheated and made this instead:

public class PassThroughStream : Stream
{
    private Stream underlyingStream;

    public PassThroughStream(Stream underlyingStream)
    {
        this.underlyingStream = underlyingStream;
    }

    public override int Read(byte[] buffer, int offset, int count)
    {
        int ix = 0;
        int left = count;
        byte[] _b = new byte[8192];
        int ch = _b.Length;

        while (left > 0)
        {
            //read a buffer. The amount will be min(amount_left_to_read,buff_len)
            var amt = underlyingStream.Read(_b, 0, left > ch ? ch : left);
            //if nothing was returned, that is all we can read.
            if (amt == 0) break;
            //copy into the buffer
            Array.ConstrainedCopy(_b, 0, buffer, ix, amt);
            //update our pointers
            ix += amt;
            left -= amt;
        }

        return ix;
    }

    /* A bunch of methods have been omitted */
}

Next time I will think twice before reading from a Stream directly... writing seems to be fine though.

EDIT: I have since discovered that 1) this code is painfully slow and 2) The BinaryReader.Read does exactly the same thing as Stream.Read. Literally, this is what it does: (decompiled source)

public virtual int Read(byte[] buffer, int index, int count)
{
    /*some error checking*/
    return this.m_stream.Read(buffer, index, count);
}

I guess the moral of the story is that Stream.Read is unreliable...

No comments:

Post a Comment