MySQL Sharding In The Cloud - Who's Your (Stack's) Daddy?

The week before last I had the privilege of attending Cloud Expo West. As usual, Santa Clara provided the perfect setting for the event with its luxurious convention center and lovely, crisp weather. It was very interesting listening to the sessions and great fun to walk the expo’s floor during the 4-day convention.

For me, one of the highlights of the expo was Cory Issacson’s session about ‘Scaling Your Database in the Cloud‘. Cory, the CEO of CodeFutures, gave the audience an excellent review of database scaling challenges in general and their manifestation in the cloud as well as how his company’s sharding solution addresses these.

The neat thing about sharding solutions is that they attempt to provide the application developer with all the benefits of database sharding whilst masking the underlying complexity of the setup. This presumably rids developers of the need for shard-aware code in the application and for sharding maintenance. Put differently, modern sharding solutions encapsulate the sharding logic that had to be coded in the application before their debut.

Naturally, for me the motivation to attend the session are the real and perceived similarities between our companies’ solutions (read competitive analysis). Coincidently, on the day before the session, I had posted my MySQL Sharding vs. MySQL Partitioning on our blog, so the topic of database sharding was still quite fresh in my mind.

It was then and there that it dawned on me that in my discussion about sharding vs. partitioning I had left out one aspect that I feel is central in the development of contemporary cloud applications, so I’d like to take this opportunity and present it. That missing aspect is stack ownership in the Cloud and its effect on the feasibility of database scaling solutions.

PaaS Solutions and Stack Ownership in the Cloud

Long ago, when the Cloud was still in its infancy, we used to deploy applications on it just like we always did – we’d take a bunch of machines and install them with everything that was needed for the application to run, including the OS, drivers, libraries, runtime components and so forth. This collection of odds and ends that’s required for the application’s normal operation is the stack. And stack maintenance was a real pain for developers.

Then PaaS (Platform-as-a-Service) was born, and allowed developers to focus on their programmatic strengths by taking care of the stack’s nitty-gritty for them.

However, by eschewing stack ownership and giving it to PaaS providers, application owners are also giving up much (if not all) of the control and flexibility that stemmed from their owning of the stack. As a developer using a PaaS you are restricted, for better and for worse, to your application’s code and in most cases you can no longer make changes to the underlying stack. Every PaaS player recognizes that this is one of the biggest challenges in providing a viable platform solution to its users and the common approach taken to addressing this is building a strong add-ons program. Add-ons are, in a manner of speaking, the way PaaS providers provide the developers to customize their stacks.

MySQL Sharding Solutions and Stack Ownership

Tying this to the original discussion about sharding and cloud applications’ development, not owning the stack is a real issue when you need to change it so it accommodates your application’s needs.

Because sharding is external to the database and because the stack is no longer yours, finding the place in the PaaS for your sharding solution can prove to be a non-trivial task when you want to replace your database driver with a shard-able one (such as CodeFutures’) or if you want to incorporate sharding middleware (such as ScaleBase’s) to your setup.

Furthermore, PaaS providers will usually have their own database services and/or database add-ons that are integral to their offering and taking the DIY approach to sharding these only compounds the complexity of the challenge.

Xeround, MySQL Scalability and the PaaS Ecosystem

Xeround offers a Database-as-a-Service (DBaaS) that can be used as a drop-in database replacement for your MySQL applications. Scaling and availability challenges are addressed internally by our database’s design and do not require any external resources or management from the application.

Regardless of the techniques that we use to achieve these goals, what our users get is a truly transparent database solution in Service form. This means that we play nicely with PaaS, whether via an add-on or not. Being a fully managed service, you can integrate Xeround’s database with your PaaS-based application just by changing its connection string and nothing more.

Original URL: http://xeround.com/blog/2011/11/mysql-sharding-in-the-cloud-and-paas