Keep Updated

Version 2.0 Benchtests – part 1

CQRS.NET 2.0, faster than ever on Azure

As part of preparing for the release of CQRS.NET 2.0 we decided to benchmark CQRS.NET in Azure using real-world equipment. This meant we didn’t want to configure specialised virtual machine with specialised configurations, ton loads of memory and cores, expensive high speed links and large caches. We wanted to see how much we could on the cheapest, easiest to use options within Azure.

The setup

Having listened to our user-base we decided to test 2 main scenarios, both with almost identical setups.

In both cases, we’re using Microsoft Azure Apps Services, hosted in Azure Functions.

The test was an extremely simple one designed to thrash the CPU via high usage of the command handlers, event handlers and IO threads. We played around with number of different combinations with the number of asynchronous tasks generating commands and the amount of commands each task generated and published.

Note that any second or third level operation below is handled for the developer by the CQRS framework. Telemetry using Azure ApplicationInsights was also enabled for all components (which does add overhead) so we were able to monitor and measure performance.

First publishing a command:

  1. Generate a random number.
  2. Construct a command and apply the randomly generated number
  3. Publish the command
    1. Validate the command
    2. Serialise the command
    3. Place the command on the command bus
      1. Seralise the network message
      2. Place the network message on the network

Receiving the command:

This starts with a non-developer initiated network message

  1. Receive the command
    1. Receive the network message
      1. Deserialise the the network message
    2. Deserialise the command
    3. Validate the command
    4. Locate the command handler
    5. Pass the command to the command handler

Responding to the command

  1. Read the random number off the command.
  2. Construct an event and apply the randomly generated number
  3. Publish the event
    1. Validate the event
    2. Serialise the event
    3. Place the event on the event bus
      1. Seralise the network message
      2. Place the network message on the network

Receiving the event:

This starts with a non-developer initiated network message

  1. Receive the event
    1. Receive the network message
      1. Deserialise the the network message
    2. Deserialise the event
    3. Validate the event
    4. Locate the event handlers (more than one)
    5. Pass the event to the event handler

Responding to the event

We did nothing here

Test 1 – In-process

For our first test we stayed completely in-process. We didn’t add an external command or event bus just to see how much we could push things using a B1 app service plan. This is the first plan to have an SLA , is extremely cheap, but also extremely limited on resourcing. For 7.5 cents an hour you get 1 core and 1.75 GB of memory to plan with. Plenty for start-ups and lean development projects. This approach also meant both commands and event are processed within the same Azure Function.

The results

Application Insights

To stay using out-of-the-box Azure features we provide telemetry and monitoring via Azure Application Insights. Here we measure a command as a request and an event as a dependency. The request duration and dependency duration is the total amount of time to handle a command or event excluding time to receive the network message and deserialise the network message, it does however include network message serialisation and time for sending the network messages.

The results showed that this simple test, could push up to 1,758 commands per second, process them via the command handlers, raise a matching event and process that via an event handler before the CPU (and probably thread context switching) started to max out. We used very little memory and had an extremely low latency around command and event handling – 1 millisecond on average.

Outcome

We’re pretty stocked with the outcome of this. Doing some basic maths we see this as:

  • 105,480 operations per minute costing 0.125 cents.
  • 6,328,800 operations per hour costing 7.5 cents.
  • 151,891,200 operations per day costing $1.80.

For a free, open sourced framework we think it’s pretty good!

We’ll post the results of going across the Azure ServiceBus soon as well as the source code.

Leave a Reply

Your email address will not be published. Required fields are marked *