RabbitMQ + .NET C#: Low and variable performances on message publishing + consuming

Question

I'm trying to improve the performances of publishing and consuming messages with RabbitMQ + .NET C#.

I've defined the relatively "big" xml file (as an embedded resource) below:



  
    
      ABC/090928/CCT001
      2009-09-28T14:07:00
      3
      11500000
      
        ABC Corporation
        
          Times Square
          7
          NY 10036
          New York
          US
        
      
    
    
      ABC/086
      TRF
      false
      2009-09-29
      
        ABC Corporation
        
          Times Square
          7
          NY 10036
          New York
          US
        
      
      
        
          
            00125574999
          
        
      
      
        
          BBBBUS33
        
      
      
        
          ABC/090928/CCT001/01
          ABC/4562/2009-09-08
        
        
          10000000
        
        SHAR
        
          
            AAAAGB2L
          
        
        
          DEF Electronics
          
            Corn Exchange 5th Floor
            Mark Lane 55
            EC3R7NE London
            GB
          
        
        
          
            
              23683707994125
            
          
        
        
          CINV
        
        
          
            
              4562
              2009-09-08
            
          
        
      
      
        
          ABC/090628/CCT001/2
          ABC/ABC-13679/2009-09-15
        
        
          500000
        
        CRED
        
          
            DDDDBEBB
          
        
        
          GHI Semiconductors
          
            Avenue Brugmann
            415
            1180
            Brussels
          
        
        
          
            BE30001216371411
          
        
        
          PHOB
          +32/2/2222222
        
        
          GDDS
        
        
          
            
              
                
                  CINV
                
              
              ABC-13679
              2009-09-15
            
          
        
      
      
        
          ABC/090928/CCT001/3
          ABC/987-AC/2009-09-27
        
        
          1000000
        
        SHAR
        
          
            BBBBUS66
          
        
        
          ABC Corporation
          
            Treasury department
            Bush Street
            13
            CA 94108
            San Francisco
            US
          
        
        
          
            
              4895623
            
          
        
        
          INTC
        
        
          
            
              
                
                  CINV
                
              
              987-AC
              2009-09-27

Then I publish that file above about 500,000 times to the same queue, publishing and consuming happen in the same .NET Core C# program below:

.NET C# Code:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Text;

using RabbitMQ.Client;
using RabbitMQ.Client.Events;


namespace CSharpPlayground
{
    public static class EmbeddedResource
    {
        public static string Load(string fileName)
        {
            var executingAssembly = Assembly.GetExecutingAssembly();
            var resource = $"{executingAssembly.ManifestModule.Name.Replace(".dll", string.Empty)}.{fileName}";
            using var stream = executingAssembly.GetManifestResourceStream(resource);
            using var streamReader = new StreamReader(stream!);
            return streamReader.ReadToEnd();
        }
    }

    public static class CheapExtensions
    {
        public static void ForEach(this IEnumerable source, Action action)
        {
            foreach(var item in source)
            {
                action(item);
            }
        }
        public static void ForEach(this IEnumerable source) =>
            source.ForEach(item => {});
    }

    public static class Program
    {
        private static readonly string XmlStuff = EmbeddedResource.Load("XmlStuff.xml");

        public static void Main()
        {
            const string queueName = "hello";
            var factory = new ConnectionFactory
            {
                HostName = "localhost",
                DispatchConsumersAsync = true
            };
            using var connection = factory.CreateConnection();
            using var queueDeclareChannel = connection.CreateModel();

            queueDeclareChannel.QueueDeclare(
                queue: queueName,
                durable: true,
                exclusive: false,
                autoDelete: false,
                arguments: null);

            using var consumerChannel = connection.CreateModel();
            var consumer = new AsyncEventingBasicConsumer(consumerChannel);
            consumer.Received += async (sender, eventArgs) =>
            {
                var receivedBody = eventArgs.Body;
                var receivedMessage = Encoding.UTF8.GetString(receivedBody.ToArray());
                //Console.WriteLine($"[x] Received stuff");
            };
            consumerChannel.BasicConsume(
                queue: queueName,
                autoAck: true,
                consumer: consumer);

            var messageToSend = XmlStuff;
            var bodyToSend = new ReadOnlyMemory(Encoding.UTF8.GetBytes(messageToSend));

            Enumerable
                .Repeat((messageToSend, bodyToSend), 500_000)
                .AsParallel()
                .Select((data, index) =>
                {
                    var (message, body) = data;
                    using var publishChannel = connection.CreateModel();
                    var basicProperties = publishChannel.CreateBasicProperties();
                    publishChannel.BasicPublish(
                        exchange: "",
                        routingKey: queueName,
                        mandatory: false,
                        basicProperties: basicProperties,
                        body: body);
                    //Console.WriteLine($"[x] Sent stuff"
                    return index;
                }).ForEach();

            Console.ReadKey();
        }
    }
}

I've tried to the extent possible to remove the Console.WriteLine calls to circumvent the bottleneck of pushing something to standard output.

It turns out that the performances seem rather low, and I'm wondering if the performances I have (in terms of message rates) are somehow "normal":

RabbitMQ + .NET C#: Low and variable performances on message publishing + consuming

Answers (1)

Related Questions