So before you invest a lot of time and energy into a particular cloud it is important to understand the overall performance characteristics of MongoDB on that cloud.  We looked for this information and did not find it – so we decided to put it together for you as part of our performance series.

The Benchmark Rig

We decided to compare AWS, Azure and DigitalOcean for this test. Two different sets of configurations were chosen. The table below summarizes the machine configurations:

* Refer here under “MongoDB Hosting” for  the details machine configurations.
**  DigitalOcean has directly attached SSDs.

The benchmark performance tests were run using YCSB Workload A (update heavy workload). We talked about YCSB, setting it up and its workloads in a very detailed post last month.

1. All benchmark tests were performed in a standalone configuration
2. For both configurations, 5 million records were inserted with various levels of server load (based on the number of client threads).
3. For the Medium configuration, then Workload A was executed with default values (50% Update, 50% Read) with operation count of 5 million at various levels of server load.
4. For the Large configuration, then Workload A was executed with default values (50% Update, 50% Read) with operation count of 10 million at various levels of server load.

Results

We will discuss results based on insert performance and throughput/latency characteristics under update heavy workload.

Insert Performance

Medium Instances

The Throughput/latency characteristics for 5M record insertion on the Medium configuration:

Large Instances

The Throughput/latency characteristics for 5M record insertion on the Large configuration:

Update Performance

Medium Instances

The Throughput/latency characteristics for 5M write/update operations on the medium configuration:

The test was run with 32 threads for DigitalOcean only. AWS and Azure were flat linings at 16 threads. However, DigitalOcean gives the impression of scaling linearly until 32 threads.

Large Instances

The Throughput/latency characteristics for 10M write/update operations on the Large configuration:

Overall Analysis

1. As expected, MongoDB on DigitalOcean has consistently high throughput/low latency characteristics throughout and beats the others hands down during the insert phase, extracting the maximum juice out of its local SSD drives. Interestingly, even though it goes very well during the read/update phase, other providers give it a fair bit of competition, especially when the server load increases. Clearly AWS/Azure are using much higher throughput networked storage.
2. In order to get better performance from AWS disk, the user can use larger disk sizes or allocate more provisioned IOPS.
3. On Medium instances, MongoDB on Azure seems to do much better than AWS consistently, both during insert and then update/read phases. This was surprising. The hardware is pretty evenly matched. On Large instances, AWS performance is markedly better than Azure.
4. Both AWS and Azure degrade in latencies pretty well as the load increases. Azure does seem to have a rather good latency degradation curve.
5. Another interesting aspect of MongoDB on AWS performance throughout is how “flat-lined” it is: Seems to degrade gracefully even on a log scale.
6. Based on latencies numbers, looks like the sweet spot, from a load standpoint, for Medium and Large instances is 8 and 16 threads respectively.