Hello!
The comments on this profile are unclaimed and thus are unverified.
Do they belong to you? Claim these comments.
_Alan_
Is this you? Claim Profile »
9 months ago
in Stream.CopyStream - Miguel de Icaza on Miguel de Icaza's blog
Well, if all you're looking for is:
stream2.Write(stream1), then that's trivial enough to implement. However if you want something more along the lines of giving you three identical copies of the same stream so you can independently read from the copies and progress the initial stream, then it's a lot harder.
byte[] bytes = GetData();
MemoryStream a = new MemoryStream(bytes);
MemoryStream b = new MemoryStream(bytes);
MemoryStream c = new MemoryStream(bytes);
If you want a CopyStream that essentially does the above except using a Stream as the base rather than a byte[] as the base, the task is very difficult.
stream2.Write(stream1), then that's trivial enough to implement. However if you want something more along the lines of giving you three identical copies of the same stream so you can independently read from the copies and progress the initial stream, then it's a lot harder.
byte[] bytes = GetData();
MemoryStream a = new MemoryStream(bytes);
MemoryStream b = new MemoryStream(bytes);
MemoryStream c = new MemoryStream(bytes);
If you want a CopyStream that essentially does the above except using a Stream as the base rather than a byte[] as the base, the task is very difficult.
1 reply
Atif Aziz
Alan, I don't see why it would this would be difficult to implement CopyStream having Stream as a source and target. Here's my implementation that I use in projects:
http://gist.github.com/12956
In general, though, this only useful for low latency and small streams. In environments where scalability is key, copying may be best performed using async I/O (BeginRead-BeginWrite).
http://gist.github.com/12956
In general, though, this only useful for low latency and small streams. In environments where scalability is key, copying may be best performed using async I/O (BeginRead-BeginWrite).
9 months ago
in Stream.CopyStream - Miguel de Icaza on Miguel de Icaza's blog
How could System.IO.Stream ever implement CopyStream? It'd be a nightmare to implement!
The problem is that the Stream in question could be *anything*, i.e. a NetworkStream. By reading data from this stream (in order to make a copy) you will render the initial stream unusable because you've already read the data out, and you cannot rewind it. What you end up with is a single usable version of the stream instead of two.
If you want to be able to 'Copy' a stream, you have to do it manually. You need to read all the data from the initial stream as a byte[], then instantiate all the MemoryStreams you want from this byte[].
Suppose you wanted to implement Stream.CopyStream, you could create a class like
public class StreamSplitter : Stream
{
public StreamSplitter (Stream initialStream)
{
// Do stuff
}
public Stream GetCopy()
{
// Do stuff
}
}
Internally the StreamSplitter will read from the initial stream and store the data in a byte[]. GetCopy will then return a MemoryStream type class based on this byte[]. This class would have the ability to tell the StreamSplitter to get more data from the initial stream and make it available to all the copies which have been created.
The problem is that any copy can initiate this request for more data at any time. This is a pain in the ass for thread synchronisation with regards to the initial stream. Then you have more threading problems when you try making this new data available to the existing copies. All i can say is that there's no way i'd ever implement something like that ;)
The problem is that the Stream in question could be *anything*, i.e. a NetworkStream. By reading data from this stream (in order to make a copy) you will render the initial stream unusable because you've already read the data out, and you cannot rewind it. What you end up with is a single usable version of the stream instead of two.
If you want to be able to 'Copy' a stream, you have to do it manually. You need to read all the data from the initial stream as a byte[], then instantiate all the MemoryStreams you want from this byte[].
Suppose you wanted to implement Stream.CopyStream, you could create a class like
public class StreamSplitter : Stream
{
public StreamSplitter (Stream initialStream)
{
// Do stuff
}
public Stream GetCopy()
{
// Do stuff
}
}
Internally the StreamSplitter will read from the initial stream and store the data in a byte[]. GetCopy will then return a MemoryStream type class based on this byte[]. This class would have the ability to tell the StreamSplitter to get more data from the initial stream and make it available to all the copies which have been created.
The problem is that any copy can initiate this request for more data at any time. This is a pain in the ass for thread synchronisation with regards to the initial stream. Then you have more threading problems when you try making this new data available to the existing copies. All i can say is that there's no way i'd ever implement something like that ;)
1 reply
migueldeicaza
Alan,
In that case, you would not use CopyStream, it is a perfectly fine restriction to say that the stream is consumed after CopyStream is finished.
There is nothing preventing a "TeeStream" (like the Unix Tee command) to duplicate the contents as it goes and composing the above described operation using this.
But many times I do not care about keeping a copy, and I do not care about chunked output, or if its network or not, I just want to move the bytes from the source to the destination. And I have seen various broken versions of this loop, or versions that are too optimistic and have never been properly tested.
In that case, you would not use CopyStream, it is a perfectly fine restriction to say that the stream is consumed after CopyStream is finished.
There is nothing preventing a "TeeStream" (like the Unix Tee command) to duplicate the contents as it goes and composing the above described operation using this.
But many times I do not care about keeping a copy, and I do not care about chunked output, or if its network or not, I just want to move the bytes from the source to the destination. And I have seen various broken versions of this loop, or versions that are too optimistic and have never been properly tested.
Was it you and Jo that sent all the poems? See the Supercomputing thread for more evidence.