Amazon EC2 (Amazon Elastic Compute Cloud) is a fabulous cloud computing platform. A majority of the internet runs on Amazon AWS – when users refer to “cloud computing” they are implicitly talking about Amazon AWS. My company has been running and managing databases on AWS for a couple of years now and we have learned a lot from our experiences. While AWS is an easy platform to get up and running, it’s extremely difficult to run large disk-intensive workloads on AWS. I’m not saying it cannot be done – however, the time and expertise required is beyond most users. Here are a few things that I’d like to see in Amazon EC2 to make it easier to run databases on AWS.
-
Non-Ephemeral Local Disks
Network-based EBS is convenient for most workloads, but performance is abysmal for write-heavy workloads. The introduction of provisioned IOPS eases this problem a little bit. However, Provisioned IOPS are fairly expensive and the costs add up, especially when you’re running a big cluster with 10-20 machines. As an alternative, it will be great if disk heavy workloads like databases could run off of the local disk. It’s not an option today because the local disks are “ephemeral”. If you stop and restart your machine, it might move to a different host and you lose your local data. This is not an acceptable risk even when there are multiple copies of data.
-
Low-Cost SSD
It would be great if Amazon can take a leaf out of DigitalOcean’s book and introduce low-cost SSD’s for its servers. Server-side computing is slowly moving to SSD and in a few years, SSD servers will be the defacto storage for your server workloads. Amazon does offer SSD’s today, but they’re fairly expensive and not an option for most workloads. Also, the SSD offering has the same “ephemeral” problem as local disks.
-
Cross-Region Security Groups
Geo-distributed clusters are a reality of our times. A number of customers need to deploy servers across regions for multiple reasons ranging from availability to partitioning. The only way to secure these deployments today is by using an IP whitelist which is extremely difficult to maintain. Cross-region security groups will greatly alleviate the burden for customers deploying across multiple regions. today, Amazon has very little functionality that works across regions. They recently introduced the ability to copy templates across regions which is very useful, and I hope they continue to add more features that are cross-region.
-
Synchronized Snapshots Across Multiple Volumes
In some of our larger database clusters, we need to backup multiple servers simultaneously. For example, In a sharded MongoDB cluster, you need to backup a consistent copy of all the shards. While there are techniques to do this today, they’re all fairly hairy and vulnerable to failure. An ideal way to backup these servers is to kick off a synchronized snapshot across several volumes. This will ensure a consistent snapshot across all the volumes.
-
Better VPC Management
I personally don’t like the idea of exposing production databases to the internet. Hence, I’m a big fan of Virtual Private Clouds (VPC). The technology is great but the management interface is fairly tedious. VPC and classic EC2 are very similar until they’re not. You end up switching back and forth between the EC2 console and the VPC console. Once you’re managing 10+ servers, the current management paradigm places a lot of burden on the user. I think there’s room to simplify the concepts and make it easier to manage.
As always, if you have questions, please feel free to reach out to us [email protected].
