File transfer issue on Mac OS

[Anonymous]

I am trying to send files over network on Mac OS. It works fine on Windows.
I use slightly modified code form this forum to send files. Can’t find it right now.
Small files ~100KB are sent just ok, but big one ~10MB give me this error:

System.TimeoutException: Write timed out after 2000ms
at NetworkCommsDotNet.Tools.StreamTools.Write (System.IO.Stream inputStream, Int64 inputStart, Int64 inputLength, System.IO.Stream destinationStream, Int32 writeBufferSize, Double timeoutMSPerKBWrite, Int32 minTimeoutMS) [0x00000] in <filename unknown>:0
at NetworkCommsDotNet.Tools.StreamTools+ThreadSafeStream.CopyTo (System.IO.Stream destinationStream, Int64 startPosition, Int64 length, Int32 writeBufferSize, Double timeoutMSPerKBWrite, Int32 minTimeoutMS) [0x00000] in <filename unknown>:0
at (wrapper remoting-invoke-with-check) NetworkCommsDotNet.Tools.StreamTools/ThreadSafeStream:CopyTo (System.IO.Stream,long,long,int,double,int)
at NetworkCommsDotNet.Connections.TCP.TCPConnection.SendStreams (StreamSendWrapper[] streamsToSend, Double maxSendTimePerKB, Int64 totalBytesToSend) [0x00000] in <filename unknown>:0
at NetworkCommsDotNet.Connections.Connection.SendPacketSpecific (IPacket packet) [0x00000] in <filename unknown>:0
at NetworkCommsDotNet.Connections.Connection.SendPacket[StreamSendWrapper] (IPacket packet, System.Int64& packetSequenceNumber) [0x00000] in <filename unknown>:0

This stacktrace doesn’t even connected to my code, because it is timeout exception. I suspect that it is caused by that code:

var connection = TCPConnection.GetConnection (new ConnectionInfo (new IPEndPoint (point.Address, point.Port)), true);

			var fileName = System.IO.Path.GetFileName (filePath);

			using (var file = new FileStream (filePath, FileMode.Open)) {
				using (var safeStream = new StreamTools.ThreadSafeStream (file)) {
					var sendChunkSizeBytes = (long)(file.Length / 20.0) + 1;

					//Limit send chunk size to 500MB
					const long maxChunkSizeBytes = 500L * 1024L * 1024L;
					if (sendChunkSizeBytes > maxChunkSizeBytes)
						sendChunkSizeBytes = maxChunkSizeBytes;

					long totalBytesSent = 0;
					do {
						//Check the number of bytes to send as the last one may be smaller
						var bytesToSend = (totalBytesSent + sendChunkSizeBytes < file.Length ? sendChunkSizeBytes : file.Length - totalBytesSent);

						//Wrap the threadSafeStream in a StreamSendWrapper so that we can get NetworkComms.Net
						//to only send part of the stream.
						using (var streamWrapper = new StreamTools.StreamSendWrapper (safeStream, totalBytesSent, bytesToSend)) {
							//We want to record the packetSequenceNumber
							long packetSequenceNumber;
							//Send the select data
							connection.SendObject ("PartialFileData", streamWrapper, NetworkComms.DefaultSendReceiveOptions, out packetSequenceNumber);
							//Send the associated SendInfo for this send so that the remote can correctly rebuild the data
							connection.SendObject ("PartialFileDataInfo", new SendInfo (fileName, file.Length, totalBytesSent, packetSequenceNumber));
							totalBytesSent += bytesToSend;
						}

					} while (totalBytesSent < file.Length);
				}
			}

It looks like different OS, have different socket/network settings/timeouts. I haven’t find at networkcomms settings to adjust that. What is the best way to resolve this situation?

[Anonymous]

A chunk size of 500MB is rather insane, no surprise you are having problems. I recommend you change the chunksize to a maxmimum of 1 or 2MB.

Also there is an excellent file transfer example included in the samples you get with NetworkComms.Net. I highly recommend you reference this as it is a comprehensive example. What you will find on the forums are only minimal ones.

Regards,
Marc

[Anonymous]

I tried as small chunksize as 100kb, it still fails with same mistake. What example you talking exactly? Advanced – Distributed File System?

[Anonymous]

As a side not DFS example fail on any platform if you select first logger, got host mode, then pick 1mb package size. Full log:

Initiating NetworkCommsDotNet examples.

To enable comms logging press 'y'. To leave logging disabled and continue press any other key.

Please select the logger to use:
1 - NetworkComms.Net LiteLogger
2 - External NLog Logger
 ... selected NetworkComms.Net LiteLogger.

15.7.36.512 [1 - Info] - NetworkComms.Net logging enabled.
Please selected an example:

1 - Basic - Message Send (Only 11 lines!)

2 - Intermediate - Message Send
3 - Intermediate - Peer Discovery

4 - Advanced - Object Send
5 - Advanced - Distributed File System
6 - Advanced - Remote Procedure Call
7 - Advanced - Unmanaged Connections
8 - Advanced - TCP (SSL) Connections

9 - Debug - Speed Test

5

Distributed File System (DFS) Example ...

Please select host or peer mode:
1 - Host Mode (Original source of data)
2 - Peer Mode (Builds data and then acts as subhost)

 ... host mode selected.

Please enter how large the test data packet should be in MB and press return, e.g. 50:
1

 ... successfully created a 1MB test packet.
15.7.41.245 [1 - Info] - Added incoming packetHandler for 'DFS_IncomingLocalItemBuild' packetType.
15.7.41.245 [1 - Info] - Added incoming packetHandler for 'DFS_RequestLocalItemBuild' packetType.
15.7.41.246 [1 - Info] - Added incoming packetHandler for 'DFS_ChunkAvailabilityInterestRequest' packetType.
15.7.41.246 [1 - Info] - Added incoming packetHandler for 'DFS_ChunkAvailabilityInterestReplyData' packetType.
15.7.41.246 [1 - Info] - Added incoming packetHandler for 'DFS_ChunkAvailabilityInterestReplyInfo' packetType.
15.7.41.246 [1 - Info] - Added incoming packetHandler for 'DFS_ChunkAvailabilityRequest' packetType.
15.7.41.246 [1 - Info] - Added incoming packetHandler for 'DFS_PeerChunkAvailabilityUpdate' packetType.
15.7.41.246 [1 - Info] - Added incoming packetHandler for 'DFS_ItemRemovalUpdate' packetType.
15.7.41.246 [1 - Info] - Added incoming packetHandler for 'DFS_KnownPeersUpdate' packetType.
15.7.41.246 [1 - Info] - Added incoming packetHandler for 'DFS_KnownPeersRequest' packetType.
15.7.41.246 [1 - Info] - Added incoming packetHandler for 'DFS_ItemLinkRequest' packetType.
15.7.41.247 [1 - Info] - Added globalConnectionShutdownDelegates. 1
15.7.41.247 [1 - Info] - Starting DFS listeners.
15.7.41.248 [1 - Fatal] - Error_DFSIntialise 3.07.41.247 30-09-2014 [88315-0]
15.7.41.261 [1 - Debug] - New swarmChunkAvailability created using System.Collections.Generic.List

1[NetworkCommsDotNet.ConnectionInfo] possible sources.
15.7.41.262 [1 – Debug] – … created new original DFS item (C801C26F2D09039E69D4FF9E9099680B).
… DFS has been initialised.
15.7.41.262 [1 – Info] – Added globalConnectionEstablishDelegates. 1
15.7.41.262 [1 – Info] – Added globalConnectionShutdownDelegates. 2
15.7.41.263 [1 – Info] – Added incoming packetHandler for ‘BigDataRequest’ packetType.
15.7.41.263 [1 – Info] – Added incoming packetHandler for ‘ClientInfo’ packetType.

Listening for incoming connections on:

Identifier – HetjvDR7LUaU7QaTX0Vjfg

Press ‘s’ to write out stats, ‘q’ to close any connected peers, ‘ctrl+q’ to close this host.`

[Anonymous]

I guess, you are probably talking about ExamplesFileTransfer.WPF?
This is the one I am using right now.
It has this code

 //Limit send chunk size to 500MB
                        long maxChunkSizeBytes = 500L*1024L*1024L;
                        if (sendChunkSizeBytes > maxChunkSizeBytes) sendChunkSizeBytes = maxChunkSizeBytes;

And it is WPF only, so you can’t test it on any other platforms, that is why it has never been tested on Mac/Linux. I guess.

[Anonymous]

The code that was used to create the ExamplesFileTransfer.WPF example came from a mono application.

I cannot see any error message in the DFS ouput you have listed.

Please submit any potential bug reports to our tracker, rather than posting on the forum. This makes it easier to deal with if they are not quick fixes – https://bitbucket.org/MarcF/networkcomms.net/issues/new

Regards,
Marc

[Anonymous]

The code that was used to create the ExamplesFileTransfer.WPF example came from a mono application.

Where is the mono application? Point was that you don’t have file transfer sample for Mac/Linux.

I cannot see any error message in the DFS ouput you have listed.

Pay attention to this line:
15.7.41.248 [1 - Fatal] - Error_DFSIntialise 3.07.41.247 30-09-2014 [88315-0]
Code fails to start to listen.

Please submit any potential bug reports to our tracker, rather than posting on the forum. This makes it easier to deal with if they are not quick fixes – https://bitbucket.org/MarcF/networkcomms.net/issues/new

Yeah, It seems this is not going to be quick.

[Anonymous]

We will do our best to fix this issue as quickly as our current schedule allows.

If you would like a quicker resolution we do offer commercial support packages.

Regards,
Marc

[Author]

Posts