System.OutOfMemoryException

Home › Forums › Support › System.OutOfMemoryException

Viewing 5 posts - 1 through 5 (of 5 total)
  • Author
    Posts
  • February 7, 2014 at 05:44 #1468
    Anonymous
    Inactive

    Hi, I would like to know how to best handle this situation. I made a socket server using the NetworkComms 2.3.0 library. The socket server expects at least 500 users to connect with continuous send and receive operations. Actually, the application is an ERP application. The problem is that the program can’t handle a lot of methods running asynchronously. Am I doing something wrong? To simulate the problem, I created a program for stress testing. The code can be found below:

    Here’s the Socket Tester:

    using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using NetworkCommsDotNet;
using System.Timers;

namespace SocketTest
{
    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    {
        public MainWindow()
        {
            InitializeComponent();
            this.Closing += (s, e) =>
            {
                NetworkComms.Shutdown();
            };
            var t = new Timer()
            {
                AutoReset = true,
                Enabled = true,
                Interval = 100
            };
            t.Elapsed += (s, e) =>
            {
                NetworkComms.SendObject("poll", "127.0.0.1", 30000, DateTime.Now);
            };

            NetworkComms.AppendGlobalIncomingPacketHandler<DateTime>("server", (packetType, connection, message) =>
            {
                App.Current.Dispatcher.Invoke(() =>
                {
                    haha.Items.Add(message);
                });
            });
        }

        private void Window_PreviewKeyDown(object sender, KeyEventArgs e)
        {
            NetworkComms.SendObject("key", "127.0.0.1", 30000, e.Key.ToString());
        }

    }
}

    Here’s the Socket Server:

    using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using NetworkCommsDotNet;
using System.Timers;

namespace SocketServer
{
    class Program
    {
        static void Main(string[] args)
        {
            NetworkComms.DefaultListenPort = 30000;
            NetworkComms.AppendGlobalIncomingPacketHandler<string>("key", (packetType, connection, message) =>
            {
                Console.WriteLine("key input from {0} : {1}", connection.ConnectionInfo.RemoteEndPoint, message);
            });

            NetworkComms.AppendGlobalIncomingPacketHandler<DateTime>("poll", (packetType, connection, message) =>
            {
                Console.WriteLine("poll received from {0} containing {1}", connection.ConnectionInfo.RemoteEndPoint, message);
            });
            TCPConnection.StartListening(true);

            var t = new Timer()
            {
                AutoReset = true,
                Enabled = true,
                Interval = 500
            };
            t.Elapsed += (s, e) =>
            {
                var list = NetworkComms.GetExistingConnection().ToList();
                foreach (var data in list)                {
                    data.SendObject("server", DateTime.Now);
                }
            };
            t.Start();
        }
    }
}

    I made like 30 – 50 instances of the Socket Tester until the Socket Server produce a .txt file containing a System.OutOfMemoryException.

    Please take a look at my codes if improvements can still be made.

    Note: About the NetworkComms.GetExistingConnection().ToList() <– I am aware that this takes memory.

    February 7, 2014 at 05:52 #1469
    Anonymous
    Inactive

    Follow Up:

    I tried removing the .ToList() from the Note above and the same thing happens after 30 instances of the Socket Tester.

    here’s the content of one of the error log:

    Base Exception Type: System.OutOfMemoryException: Exception of type ‘System.OutOfMemoryException’ was thrown.
    at LZMA.LZ.OutWindow.Create(UInt32 windowSize) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\SevenZipLZMACompressor\LZ\LzOutWindow.cs:line 36
    at LZMA.Decoder.SetDictionarySize(UInt32 dictionarySize) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\SevenZipLZMACompressor\LzmaDecoder.cs:line 196
    at LZMA.Decoder.SetDecoderProperties(Byte[] properties) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\SevenZipLZMACompressor\LzmaDecoder.cs:line 383
    at LZMA.SevenZipHelper.DecompressStreamToStream(Stream inStream, Stream outStream) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\SevenZipLZMACompressor\SevenZipHelper.cs:line 294
    at SevenZipLZMACompressor.LZMACompressor.ReverseProcessDataStream(Stream inStream, Stream outStream, Dictionary`2 options, Int64& writtenBytes) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\SevenZipLZMACompressor\LZMACompressor.cs:line 75
    at DPSBase.DataSerializer.DeserialiseGeneralObject[T](MemoryStream receivedObjectStream, List1 dataProcessors, Dictionary2 options) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\DPSBase\DataSerializer.cs:line 243
    at DPSBase.DataSerializer.DeserialiseDataObject[T](MemoryStream receivedObjectStream, List1 dataProcessors, Dictionary2 options) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\DPSBase\DataSerializer.cs:line 237
    at NetworkCommsDotNet.PacketTypeHandlerDelegateWrapper`1.DeSerialize(MemoryStream incomingBytes, SendReceiveOptions options) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\NetworkCommsDotNet\Tools\PacketUnwrappers.cs:line 72
    at NetworkCommsDotNet.NetworkComms.TriggerGlobalPacketHandlers(PacketHeader packetHeader, Connection connection, MemoryStream incomingDataStream, SendReceiveOptions options, Boolean ignoreUnknownPacketTypeOverride) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\NetworkCommsDotNet\NetworkComms.cs:line 1252

    Stack Trace: at LZMA.LZ.OutWindow.Create(UInt32 windowSize) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\SevenZipLZMACompressor\LZ\LzOutWindow.cs:line 36
    at LZMA.Decoder.SetDictionarySize(UInt32 dictionarySize) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\SevenZipLZMACompressor\LzmaDecoder.cs:line 196
    at LZMA.Decoder.SetDecoderProperties(Byte[] properties) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\SevenZipLZMACompressor\LzmaDecoder.cs:line 383
    at LZMA.SevenZipHelper.DecompressStreamToStream(Stream inStream, Stream outStream) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\SevenZipLZMACompressor\SevenZipHelper.cs:line 294
    at SevenZipLZMACompressor.LZMACompressor.ReverseProcessDataStream(Stream inStream, Stream outStream, Dictionary`2 options, Int64& writtenBytes) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\SevenZipLZMACompressor\LZMACompressor.cs:line 75
    at DPSBase.DataSerializer.DeserialiseGeneralObject[T](MemoryStream receivedObjectStream, List1 dataProcessors, Dictionary2 options) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\DPSBase\DataSerializer.cs:line 243
    at DPSBase.DataSerializer.DeserialiseDataObject[T](MemoryStream receivedObjectStream, List1 dataProcessors, Dictionary2 options) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\DPSBase\DataSerializer.cs:line 237
    at NetworkCommsDotNet.PacketTypeHandlerDelegateWrapper`1.DeSerialize(MemoryStream incomingBytes, SendReceiveOptions options) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\NetworkCommsDotNet\Tools\PacketUnwrappers.cs:line 72
    at NetworkCommsDotNet.NetworkComms.TriggerGlobalPacketHandlers(PacketHeader packetHeader, Connection connection, MemoryStream incomingDataStream, SendReceiveOptions options, Boolean ignoreUnknownPacketTypeOverride) in c:\Users\Karnifexx\Documents\Visual Studio 2010\Projects\networkcomms.net\NetworkCommsDotNet\NetworkComms.cs:line 1252

    February 7, 2014 at 08:54 #1470
    Anonymous
    Inactive

    Follow up:

    Code modified, further testing confirms that the ‘SendObject’ function is the culprit. It takes a lot of memory especially in the foreach loop.

    February 7, 2014 at 13:24 #1471
    Anonymous
    Inactive

    Thanks for posting your issue. The OutOfMemoryException occurs because the compression stage, which is enabled by default, is a fairly memory intensive operation.

    You should be able to avoid this problem by disabling the compression on both client and server. Simply add this line to the start of your applications:

    NetworkComms.DefaultSendReceiveOptions = new SendReceiveOptions<ProtobufSerializer>();

    I would also recommend caching the connection list on the server:

    var list = NetworkComms.GetExistingConnection().ToList();

    and only updating the cache every 5 seconds or so. This will greatly increasing the performance of the polling.

    If you have any further questions please feel free to post back.

    Kind regards,
    Marc

    February 8, 2014 at 02:40 #1473
    Anonymous
    Inactive

    Thanks, that sorted everything. 🙂

  • Author
    Posts
Viewing 5 posts - 1 through 5 (of 5 total)
  • You must be logged in to reply to this topic.
Blog Categories
Blog Tutorials