Are you hosting your production MongoDB instances on Amazon AWS? At ScaleGrid, we manage hundreds of production MongoDB instances on AWS and have learned a few things along the way. Here is a set of 10 questions you need to ask yourself and answer as you continue to manage your deployment. Almost all of the information below is applicable to other cloud service providers as well.
What is your high availability (HA) plan?
If you’re using a single instance, it might be time to look at replica sets. When using replica sets, take care to ensure that you deploy each replica set in a different availability zone.
What is your disaster recovery (DR) plan?
If you’re deploying all of your replica sets in one region, what happens when an entire AWS region melts down as it happened in April 2011? You might want to look into distributing your replica sets across regions.
Have you tested your DR plan?
Simulate machine, network and disk failures to understand your cluster behavior under failure conditions. You don’t want to encounter your first failover in production.
Are you backing up your instances?
Yes, you need backups even if you have replica sets. Backups are necessary to deal with accidental erasure or when a new version of your app corrupts all your data. Make sure you are backing up regularly – preferably every few hours. You can backup from the secondary so that there is not a big impact on the primary. For a general overview of MongoDB backup options, refer to our blog post - MongoDB backup options
Do your backups work?
Have you tried a recovery? How long does it take to recover and have all your replicas re-synced? If you don’t know the answer, now is a good time to do a dry run and try an end-to-end recovery.
How do you test application upgrades with production data?
One of the trickiest parts of application upgrades is testing with existing production data. Build a sequestered production like environment in which you can test your application upgrades with production data.
What sort of EBS volumes are you using?
If you’re using standard EBS volumes, consider switching to the new provisioned IOPS volumes. It’s a little bit more expensive, but worth every penny. You will see a lot less fluctuation in IO performance and sleep easier throughout the night.
Have you benchmarked the performance of your MongoDB instances?
If you haven’t, you can benchmark using mongoperf or the Yahoo cloud server benchmark (YCSB). It’s good to know what your getting from your databases.
How do you monitor your instances?
If you are not monitoring your instances, now would be a good time to start. 10gen has a freely available Mongo Monitoring Service (MMS) that you can start using to monitor your MongoDB clusters.
Are you exposing to your databases to the internet?
Todays powerful CPU and password cracking tools will crack open your password in a matter of a few hours. Use Amazon security groups to lock down access to your database and only give your front/mid-tier access to the DB.
At ScaleGrid, we’ve helped answer a number of these questions for our customers and wanted to share these tips to help you manage your MongoDB deployments on AWS. We provide a single-click deployment of MongoDB replica sets across availability zones or regions, have an automated backup and recovery process, only use provisioned IOPS and provide easy ways for our customers to benchmark MongoDB and also simulate failover in MongoDB clusters. If you have other questions/comments or feature requests we would love to hear from you. You can email us at firstname.lastname@example.org.