AppSync CloudFormation Scaling Revisited

build it

May 16, 2024

Some years back, I was building an AppSync API with a few other engineers. None of us had GQL experience and we learned as we went. At the time, I ran across an article from Yan Cui titled "How I scaled an AppSync project to 200+ resolvers"1 and thought "That's interesting, but we won't need that."

I was wrong.

We quickly started bumping up against CloudFormation's stack limit and had to implement every one of Yan's good ideas in his article. This got us out of a jam and I am grateful that he took the time to write and share with the rest of us (Cheers, Yan! 🥂).

Yan's article closes with "If you can think of a better way to do this, then please let me know!" and, until recently, I didn't know of any other options. That changed recently when I began work on a new AppSync API for another client. I realized that I could take advantage of a new AppSync feature called "Merged APIs" to distribute my stack resources in a different way than the one Yan described, and I want to share it with you here.

The Problem

Before I get into any solutions, let's try to fully understand the problem.

CloudFormation (CFN) is AWS's Infrastructure as Code (IaC) provisioning tool. It lets you define the AWS components you want to spin up and then handles the creation/update of those components (or, in CFN terms, "Resources") in a declarative way. You communicate to CFN via templates (JSON or YML) that you can store in source control and CFN "builds the specified services in [a] safe, repeatable manner."2

The number of Resources any one CloudFormation service (or "Stack) is capped. At the time of Yan's article, that cap was 200. It has since moved to 500, but there still is a cap and you cannot increase it. Yan's article describes a way to split your one service into multiple CFN Stacks such that you can distribute your resources between them and stay under the Resource cap.

Splitting stacks is complicated. Even using help from tools like serverless-plugin-split-stacks, you may have to manage how the tool splits up the stack. This "how" became the basis for Yan's article.

Another Approach

In May of 2023, AWS released Merged APIs for AppSync. This lets you create separate, discrete AppSync APIs and then present them as one unified API. The use case for this is often centered around having multiple teams work on domain-focused AppSync APIs, like so:

source: Introducing Merged APIs on AWS AppSync

In this arrangement, each independent AppSync has its own CloudFormation Stack. This distributes the number of total Resources among each of these domain-bounded Stacks. In the image, above, our overall resource cap went from 500 (for a single CFN stack) to 2000.

These two methods (Merged APIs and Split Stacks) are not mutually exclusive. For example, the Books Service may have so many resources that you need to split it. Assuming you couldn't further split at the domain level, you could apply serverless-plugin-split-stacks to just this one service to alleviate its CFN Resource cap pressure.

I have previously written about how to implement AppSync Merged API as Code, so refer to that article if you need specifics on how to wire it up.

Some Caveats

Like Yan stated in his article, I am also a one-man team for this backend. With my approach, you will end up with one serverless.yml for each separate AppSync. For example, given the domains listed in the image above, you would have five serverless manifests (one for each domain + merged): serverless.users.yml, serverless.authors.yml, serverless.books.yml, serverless.reviews.yml, and serverless.merged.yml3. You are now splitting your schema definitions and implementation instructions by domain, but I like this.

AWS initially advertised Merged APIs in AppSync as a way to address "challenges related to multiple teams"4. While I think this is a great solution for multiple teams, I don't have that problem (Yan didn't, either). Irrespective, the domain separation is something that I would do anyway; the separate deploys seems like a natural extension of that domain separation. I still have everything in a single repository, but I could easily separate it should I need to assign ownership to other teams.

Summary

Now you have another tool to help avoid the CloudFormation resource cap. If you haven't yet worked with Merged APIs, this may be the thing that lets you get your hands dirty. I find splitting up my stacks this way (by domain) to be much easier to manage than with a stacks-map.js file and the split-stacks plugin. Give it a try, and have fun!

Ownership Matters

Discussion about this post