Case Study

1 Introduction

1.1 What is BAM!?

BAM! is a serverless framework that makes it quick (hence, the name) and easy to get small applications up and running using Node.js and Amazon Web Services (AWS). It is optimized for the deployment of AWS Lambda functions integrated with Amazon API Gateway endpoints, but also allows for the creation of Amazon DynamoDB tables, which can be used to persist data between lambda invocations.

In comparison to other technologies, the biggest benefits of serverless are, arguably, the rapidity of iteration and the abstraction of infrastructure management. The irony is that the experience of working with serverless technologies often lacks the very simplicity the idea of serverless computing promises to deliver. As a result, BAM! prioritizes speed and convenience for the developer.

This article will examine the design decisions we made while building the BAM! framework, as well as our solutions to a variety of challenges we faced.

2 Serverless

The essential idea behind serverless is about where application code lives. It is easy to assume that with serverless there are no servers; however, this is a false assumption as “serverless” is a misnomer. There are physical servers somewhere in the world; they have been abstracted away so the developer does not have to manage them. Instead, the code is hosted by a third party service, which takes care of all server maintenance. For this reason, serverless has become an attractive choice for application development.

There is, however, a tradeoff when using serverless technologies. While the developer does not need to worry about managing servers, they do give up control over how their code is handled and, to some extent, how it is structured. It is important to recognize that serverless, although new and popular, is just one option among a variety of server management choices, each with their own advantages and disadvantages.

Looking at the table above, note that the amount of control decreases as the level of abstraction increases.

To start, there are “bare metal” or physical, on-premise servers. With this option, the developer has the most control with the least amount of abstraction. Next, there is IaaS, “Infrastructure as a Service”, where the developer does not have to pay the upfront cost of owning and managing physical equipment, but is still required to manage the operating system, networking, firewalls, and security of a virtual machine. Alternatively, there is CaaS, “Containers as a Service”, with which the developer maintains a certain level of flexibility through orchestration of encapsulated containers without having to manage the operating system. Depending on the situation, it may make sense to abstract away even more server responsibility. In this case, the developer could use PaaS, or “Platform as a Service”. With PaaS, the web application lifecycle is handled by the platform so the developer only has to be concerned with the application code.

These options offer quite a range of choices, with serverless offering the least amount of control, while being easiest to manage. Serverless is actually a broad category that encapsulates two others: BaaS (Backend as a Service) and FaaS (Functions as a Service).

2.1 Backend as a Service

BaaS involves multiple components pivotal to the operation of the back end. There are many different types of BaaS used to accomplish a variety of backend tasks, such as storage (e.g. AWS S3) and authorization (e.g. AuthO).

I want to outsource backend services so I can focus on the main functionality of my application.

- BaaS user

2.2 Functions as a Service

FaaS are small pieces of code that live on ephemeral containers and are usually written by the person or organization deploying them. AWS supports FaaS with AWS Lambda.

I want to write code that accomplishes a small task and can be run without worrying about scale or infrastructure.

- FaaS user

Being a type of serverless technology, FaaS requires no maintenance, which in and of itself is a huge advantage; however, this is not its only benefit. Because individual modules of code can be swapped out for other modules, iterated upon, and re-deployed, using FaaS allows for experimentation and rapid development. Additionally, FaaS is ideal for variations in traffic due to its automatic provisioning of resources.

2.3 Why AWS?

Developers can enjoy all of the above benefits by using AWS, but Amazon is by no means the only provider of serverless technologies. There are many others such as Microsoft, Google, and IBM; however, with over 1 million active customers, AWS currently enjoys prominence among engineers working with the cloud¹. We therefore built BAM! for use with AWS.

2.4 Relevant Services

AWS provides a veritable litany of cloud based services; however, we designed BAM! to interact with four in particular: AWS Lambda, Amazon API Gateway, AWS IAM, and Amazon DynamoDB.

2.4.1 AWS Lambda

Introduced in November 2014, AWS Lambda is Amazon’s take on FaaS solutions, providing an event-driven, serverless computing platform, which runs functions in response to events while automatically handling the provisioning of required resources². To accomplish this, AWS will spin up a new server instance within a container as needed, and tear down idle instances after a period of inactivity to avoid waste.

Using AWS Lambda, it is possible to code some small piece of functionality on one’s local machine and deploy it to the cloud, whereafter it can be triggered by events such as a call to an API endpoint from anywhere in the world. BAM! was built to make this process easier. While AWS Lambda supports a variety of languages and runtimes, BAM! supports Node.js 10.

2.4.2 Amazon API Gateway

Far and away one of the most common event sources encountered in cloud computing is the hitting of an API endpoint³, and the creation, management, and hosting of APIs is precisely what Amazon's API Gateway service was designed to handle.

As the name suggests, API Gateway most often acts as the outward, app-facing component of Amazon’s cloud infrastructure. Engineers can use it to build anything from a fully RESTful, secure backend to a slim, lightweight interface for interacting with a single service.

BAM! was designed specifically in the latter context of generating API Gateway endpoints that are integrated with AWS Lambda functions.

2.4.3 AWS IAM

One common thread permeating the whole of Amazon’s cloud infrastructure is the necessity for managing access permissions between various cloud services, which is where the AWS Identity and Access Management (IAM) service comes into play.

IAM allows developers to securely control authentication and authorization for use of resources at a highly granular level. For instance, one can use this service to permit a user to invoke all lambdas associated with an account, but only perform read operations on a database table.

2.4.4 Amazon DynamoDB

One final service BAM! supports is DynamoDB, Amazon’s NoSQL key-value and document storage service. We selected DynamoDB because its lack of rigid schema is a natural means of providing flexibility to users.

The services mentioned above are by no means an exhaustive list, but rather a brief discussion of those most relevant to the BAM! framework.

2.5 Challenges of AWS

While AWS is an industry leader in the new and exciting field of serverless computing, using its services is certainly not without its share of difficulties. Chief among these is a high degree of complexity, which steepens the learning curve for developers new to the technology and can prove to be burdensome in a more seasoned engineer’s day-to-day experience with the platform. There are over one-hundred fifty AWS services⁴, each with its own configuration and jargon the developer needs to be familiar with. Even more, many tasks that seem discrete are actually composed of several micro-tasks under the hood.

We will be walking through an example to see just how complex working with AWS can be.

For context, here is the scenario:

Suppose a co-worker wants to view company data, such as a sales report, and has come to you, the developer, for help. While you could manually access this data and send it to them, or write a script to produce the relevant data and manually distribute it, these options can be tedious, especially if you are frequently asked to retrieve a variation of this information. Instead, with the AWS software development kit (SDK), you can create a lambda function integrated with an API gateway. The lambda will have access and logic to process the data and the endpoint will be configured to call the lambda and display the output in a browser. This way, you will be able to give your co-worker the endpoint and enable them to retrieve the data at their convenience without disrupting your workflow.

2.6 How to manually create an AWS Lambda integrated with Amazon API Gateway

Before you can even send a request to AWS, it is necessary to perform several local operations to prepare a deployment package for the lambda. This includes confirming your AWS credentials are properly set up; deciding what configuration details to use (e.g. profile and region); creating a properly formatted, local Node.js file for the lambda function, which should contain exports.handler; installing any node package dependencies; zipping all relevant files into a deployment package; and verifying the lambda name does not conflict with any existing resources.

Now, the deployment package is ready to be sent to AWS. Start by using the AWS Lambda createFunction() method. This method takes several parameters including the zip file you just created and an IAM role. The lambda must assume an existing IAM role, which determines what permissions the lambda will have to interact with other AWS services. Then, create an API Gateway object using the createRestApi() method. It would be simple and intuitive if those were the only two steps, but there are still several more steps to complete before a callable endpoint exists.

At this point, the API Gateway will only consist of a root resource representing the root path of the API. If you want your gateway to support access to path parameters within the lambda function code, you must create an additional resource for the greedy path. The greedy path matches the path portion of the URL after the root slash (/). To accomplish this, call the Amazon API Gateway method createResource(). Even this operation is complex in that you need to retrieve the resource ID of the API’s root resource using the getResources() method and supply it as a parameter to the createResource() method.

To review, now the lambda, API gateway, and path resources exist. The following three steps are needed to add each HTTP method to the root path and integrate the API resource and associated methods with the lambda. Note: these steps will need to be repeated for the greedy path (/{proxy+}).

First, use the AWS Lambda addPermission() method to give permission to the API Gateway so that it can invoke the lambda function. Next, use the Amazon API Gateway putMethod() method to add the HTTP method to the resource. Finally, call the putIntegration() method to complete the integration between the lambda and the resource.

Generally speaking, integrations provide a way to take an incoming HTTP request to an API gateway and pass it along to some other AWS service, possibly with some intermediate processing. There are a number of integration types, including AWS, HTTP, HTTP_PROXY, MOCK, and AWS_PROXY, also known as "Lambda Proxy". If you want to expose as much of the HTTP request data as possible to your lambda, you will want to choose Lambda Proxy.

As mentioned before, the integration will need to be repeated for each HTTP method on the greedy path.

Finally, call the createDeployment() method to bundle all of these resources, methods, and integrations into one deployment resource.

Only now is the rest API endpoint ready to be called.

3 Frameworks

As you can see from the above, working with AWS directly is usually a complex process. A considerable amount of knowledge about each service is needed, and it can require some digging to determine the sequence of commands to achieve a desired outcome.

Many frameworks have come into being to enable developers to make the most of AWS while avoiding parts that are cumbersome. These frameworks range from those that are optimized for a specific use case with relatively simple capabilities to those that are harder to work with, but offer greater functionality and control.

Below, we explore four open-source frameworks in depth including BAM!.

3.1 Serverless Framework

Serverless Framework is a suite of applications that enables developers to deploy and manage serverless code. It has been around since 2015, and it is the dominant platform in this space.

Pros:

• supports a variety of providers and runtimes
• supports a wide range of configuration and functionality
• includes prefab lambda code and helpful lambda templates

Cons:

• requires YAML file configuration
• creates many AWS resources to accomplish even a simple deployment
• takes approx. 2 mins on avg for initial deployment based on our tests

3.2 Claudia.js

Claudia.js is a serverless deployment tool, which was made specifically to help JavaScript developers create AWS serverless applications easily and independently from AWS CloudFormation, Amazon's service for provisioning resources.

Pros:

• supports a wide range of configuration
• includes optimized modules for specific use cases (claudia-api-builder, claudia-bot-builder)
• is simpler than Serverless Framework

Cons:

• relies upon npm init to create a package.json
• requires separate installation of claudia-api-builder and claudia modules
• supports only Node.js runtime and AWS services
• takes approx. 1 min 15 secs on avg for initial deployment based on our tests

3.3 Shep

Shep, which bills itself as a way to “make serverless simple”, is an opinionated framework designed to streamline deployment of AWS Lambda functions that can be invoked via Amazon API Gateway endpoints.

Pros:

• uses prompts instead of configuration files
• can deploy multiple lambas at one time

Cons:

• permits few configuration options
• requires grouping code into “projects” and bundling with Webpack
• supports only Node.js runtime, AWS Lambda, and Amazon API Gateway
• takes approx. 2 mins on avg for initial deployment based on our tests

In a nutshell, Serverless is full-featured and offers many options, but it is geared toward enterprise applications and can often feel as complex as AWS itself. Claudia.js still has a lot of functionality, and though it’s simpler than Serverless, it’s not quite ideal for getting a small application up and running quickly. On the other end of the spectrum, Shep is functional, but lacks conveniences that would make working with AWS Lambda easy.

3.4 BAM!

The goal with BAM! is to have the right functionality for a small application, while being quick and easy to use for the developer.

Pros:

• does not require a “project” directory or configuration files
• can persist data between Lambda invocations via DynamoDB
• includes helpful lambda templates
• takes approx. 20 secs on avg for initial deployment based on our tests

Cons:

• permits few configuration options
• supports only Node.js runtime, AWS Lambda, Amazon API Gateway, and DynamoDB

As an example of what BAM! can do, the GIF below shows the single BAM! command (bam deploy myLambda) to deploy a lambda with an integrated endpoint:

4 BAM! Design

BAM! is designed to be human-friendly. We spent considerable time deciding which AWS services to integrate with Lambda and how to make working with those services most helpful for a developer. According to a 2018 report by Serverless Framework³ , HTTP endpoints account for more than 2/3 of all event sources. This is why BAM! is centered around lambdas connected to endpoints. As we’ve shown above, the deployment process is deceptively complex. We aimed to simplify common scenarios for developers using these services.

Our objective was to utilize an architecture that allows the developer to get up and running quickly. BAM! has flexible commands, requires no configuration, supplies instructional templates, and adapts to the developer’s local lambda file organization.

4.1 Architecture

When a BAM! command is first issued, a hidden .bam directory is created. This directory acts as a staging area for package.json file creation, dependency installments, file compression, and lambda deployment. Additionally, this directory contains a number of JSON files, which keep track of the resources deployed using the BAM! framework.

If you've used BAM! to deploy the lambda together with an API Gateway (to provide data for your co-worker, for example) the following topology will be generated to process an HTTP GET request sent to the endpoint.

There are several types of endpoints, and in most cases, the best is an Edge-optimized endpoint because AWS routes the user’s request through an AWS Cloudfront Edge Location (data center). With Cloudfront, the endpoint can be called from locations around the world with better performance.

Cloudfront will route the user's request to Amazon API Gateway, which checks the IAM role and associated policies to confirm the path resource has permission to invoke the lambda. If API Gateway receives a successful response from IAM, the lambda will be invoked to perform some processing. This could involve interacting with other services such as a database or even another lambda. In this example, the lamdba should produce a sales report, so a database query is made for last month’s total sales.

Then, the data is sent back to AWS Lambda for further processing, metadata can be persisted to a DynamoDB table, and the response is returned to API Gateway. With Lambda Proxy integrations, the lambda must return a JSON formatted response which the gateway can transform into an HTTP response. Finally, API Gateway routes the response back to the Edge location and to your co-worker, who can view the sales report in their browser.

4.1.1 A Note About Security

Edge-optimized endpoints created by AWS are public by default. Of course, if BAM! is being used to host a public webpage, this is not a concern, since the API Gateway should obviously be public. We built BAM! assuming the developer or developer’s company will take appropriate measures to secure their AWS resources in the way that best serves their needs.

4.2 Commands

The deploy command creates a lambda called <resourceName> and an API Gateway endpoint (also called <resourceName>) that can trigger the lambda.

If the lambda is coded to interact with a DynamoDB table, --permitDb causes the lambda to assume an IAM role that permits SCAN, PUT, GET, and DELETE operations. If this option is not used, the lambda assumes the default role, which is created if it does not already exist.

It is also possible to specify a role only for the current deployment using the --role flag.

If there is no need for an endpoint, --lambdaOnly deploys just a lambda.

The redeploy command updates a lambda and/or its associated endpoint.

The --role and --permitDb options work with redeploy in a similar fashion to how they work with deploy. Additionally, --revokeDb allows the lambda to assume the default role instead of the role designed to work with DynamoDB.

The --methods flag adds HTTP methods to the endpoint, and --rmMethods allows for the removal of methods attached to the endpoint.

The --addEndpoint flag adds an endpoint to a lambda that is not currently integrated with one.

The --runtime flag updates the runtime a lambda uses, however, only to an actively-supported Node runtime.

The create command creates a local file or directory based on a template, which makes it easy to quickly write code for a lambda.

There are six template options that can be created using combinations of the --invoker, --html, and --db flags. For additional guidance, --verbose can be used to create versions of these templates with instructional comments.

The list command uses data from the aforementioned hidden JSON files and AWS to log existing lambdas, associated endpoints, and DynamoDB tables, allowing quick visibility to resources in the cloud (see below).

The --dbtables and --lambdas flags can be used to log only resources for those respective services.

Lambdas and endpoints deployed from this machine using BAM!:
  nameOfLambda1
    description: a description of the lambda
    endpoint: http://associatedEndpoint/bam
    http methods: GET

  nameOfLambda2
    endpoint:: http://associatedEndpoint/bam
    http methods: GET, POST, DELETE

Other lambdas on AWS
  anotherLambda
  yetAnotherLambda

DynamoDB tables deployed from this machine using BAM!:
  nameOfTable1
    partition key: id (number)

  nameOfTable2
    partition key: id (number)
    sort key: name (string)
        

The get command pulls existing lambda code from AWS into a local directory for testing, editing, and updating.

The dbtale command creates a DynamoDB table on AWS.

The delete command deletes both the lambda and endpoint from AWS.

The --endpointOnly option can be used to delete only the endpoint, and --dbtable option can be used to delete a DynamoDB table instead of a lambda and endpoint.

The config command prompts the user to update the account number and default role.

4.3 Automatic Configuration

BAM! is an opinionated framework that allows for deployment of AWS resources without having to deal with JSON and YAML files.

Instead, BAM! finds the necessary information needed to call various SDK methods automatically. For instance, BAM! calls the AWS Security Token Service's (STS) getCallerIdentity() method in order to get the developer’s account number. Additionally, BAM! uses the region and default profile specified in the hidden config and credentials files, which exist in the .aws directory after proper installation of the AWS command line interface (CLI). Lastly, BAM! creates a default role with permissions to interact with CloudWatch logs and invoke other lambdas.

Note that since BAM! uses the aws-sdk, the framework does not directly touch the developer’s AWS Access Key or Secret Access Key.

There are several additional settings BAM! uses on behalf of the developer. These configurations are specifically for working with AWS API Gateway and include a stage and default HTTP method. An Amazon API Gateway stage specifies a set of resource-specific configuration details for the gateway. All endpoints deployed using the BAM! framework will be deployed to the bam stage. Furthermore, by default, BAM! will create an API gateway with the GET HTTP method, however, additional methods can be specified.

4.4 Templates

The BAM! framework includes six templates to help a developer get going quickly. These templates ensure lambda functions will be compatible with both the AWS Lambda programming pattern for Node.js and Lambda Proxy integration with API Gateway. Templates can be created with or without instructional comments, and although useful, these templates are not a requirement for deploying a lambda with the BAM! framework.

The first is a basic template, which is created by default when bam create is run without any flags. This template shows the developer how to handle any or all of the HTTP methods in one lambda function and exposes query and path parameters. All of the remaining templates extend this one.

There is the invoker template, which exposes the necessary parameters and payload response required to invoke another lambda. There is the HTML template, which allows a developer to write HTML, JavaScript, and CSS that can be accessed from the lambda. The next template is a combination of the invoker and HTML templates and is created simply by using the --invoker and --html flags together with the create command. The database template includes the SCAN, PUT, GET, and DELETE methods and parameters for querying a DynamoDB table. Note: to use this template properly, the lambda also needs the correct database permissions, which can be added with --permitDb flag during deployment or redeployment. The final template combines the database and HTML templates by using the --db and --html flags together with the create command.

Below is an example of the template that will be created when issuing bam create lambdaName --html --verbose.

// Welcome to your BAM! lambda!

// TODO: describe your lambda below:
// description:

const fs = require('fs');
const { promisify } = require('util');
// all require statements for npm packages should be above this line

// handler is the name of the function being exported; it's best to leave as the default
exports.handler = async (event) => {
  const { pathParameters, queryStringParameters, httpMethod } = event;

  // pathParameters will contain a property called "proxy" if path params were used
  const pathParams = pathParameters ? pathParameters.proxy : '';

  // example use of queryStringParameters to obtain value for "name" parameter
  // const name = queryStringParameters ? queryStringParameters.name : 'no name';

  const response = {};

  // it's only necessary to handle the methods you have created
  // for this API Gateway endpoint (default is GET),
  // but this is an example of how to handle
  // the response for multiple methods
  if (httpMethod === 'GET') {
    // return value must be proper http response
    response.statusCode = 200;
    // content-type headers should be set to text/html
    response.headers = { 'content-type': 'text/html' };
    // root directory of lambda function on AWS
    const rootDir = process.env.LAMBDA_TASK_ROOT;
    const readFile = promisify(fs.readFile);

    // index.html from the rootDir directory
    // note: index.html must be in rootDir directory to be accessible here
    let html = await readFile(`${rootDir}/index.html`, 'utf8');
    // main.css from the rootDir directory
    // note: main.css must be in rootDir directory to be accessible here
    const css = await readFile(`${rootDir}/main.css`, 'utf8');
    // application.js from the rootDir directory
    // note: application.js must be in rootDir directory to be accessible here
    const js = await readFile(`${rootDir}/application.js`, 'utf8');

    const replacePlaceHolder = (nameOfPlaceHolder, newText, replaceAll = false) => {
      if (replaceAll) {
        const regex = new RegExp(nameOfPlaceHolder, 'g');
        html = html.replace(regex, newText);
      } else {
        html = html.replace(nameOfPlaceHolder, newText);
      }
    };

    // there should be an empty style tag in your
    // html file that you fill with the contents of your css file
    replacePlaceHolder('<style></style>', `<style>${css}</style>`);
    // there should be an empty script tag in your
    // html file that you fill with the contents of your js file
    replacePlaceHolder('<script></script>', `<script>${js}</script>`);
    replacePlaceHolder('Placeholder', 'data from your database');

    // what the page will show
    response.body = html;
  } else if (httpMethod === 'POST') {
    response.statusCode = 201;
  } else if (httpMethod === 'DELETE') {
    response.statusCode = 204;
  } else if (httpMethod === 'PUT') {
    response.statusCode = 204;
  } else if (httpMethod === 'PATCH') {
    response.statusCode = 204;
  }

  return response;
};
        

4.5 Flexible Local File Organization

The BAM! framework allows the developer to organize lambda files in any way. For example, all lambda files could exist within one directory or could be organized into specific project directories. Because of the way BAM! is designed, the framework can handle either scenario.

Suppose a developer is deploying a lambda function they have written in Node.js, which requires the fs native Node module and the uuid npm package. After copying the lambda file or directory to the staging area, BAM! parses the require statements within the file, determines which dependencies are native to Node (in this case fs), creates a package.json file, adds only the non-native dependencies to it (in this case uuid), installs modules, zips all the files together, and deploys the lambda to AWS.

The advantages of designing the BAM! framework this way are:

• the developer’s local files remain uncluttered
• the deployment package sent to AWS is lightweight instead of being bloated with unnecessary dependencies
• the developer can ultimately deploy a lambda quickly without being well versed in the nuances of AWS services

5 Challenges

Of course, as with any engineering project the process of designing BAM! came with a host of challenges, of which the most interesting are discussed below.

5.1 Compatibility

First, it is notable that BAM! exists in a wider ecosystem, and we had to account for the possibility that users of the framework would also interact with AWS through means beyond our control. In practice, this meant that there were a plethora of conceivable edge cases where, say, the existence of a necessary resource could not be taken for granted.

5.1.1 Redeployment and API Restoration

For example, consider the case when a user has previously deployed an integrated lambda and API endpoint, and thereafter attempts to add additional HTTP methods for the API to support.

The challenge here is that unbeknownst to BAM!, the user could, meanwhile, have deliberately or accidentally deleted the API Gateway in the AWS console, while leaving the lambda intact. Under such circumstances, there will exist a lambda without an associated endpoint in the cloud. In this case, BAM! will still maintain local records of both the lambda and endpoint. If any attempt is made to update HTTP methods, the SDK will raise an exception, as there is no such API to which those methods can be attached.

We accounted for this problem by first having BAM! check for the existence of the lambda and API, and respond intelligently to the circumstances. The presence of added HTTP methods signals to BAM! the presumed existence of an API Gateway instance; BAM! will therefore create a new API behind the scenes and add the desired methods, overwriting the old local record with the replacement API’s identifying information.

In fact, BAM! will attempt to respond in a convenient and expected manner under a variety of differing circumstances when redeploying a lambda. To add a new API when one does not exist in the cloud, BAM! accepts either the --addEndpoint flag, or, to reiterate, the user may imply the addition of an API by adding HTTP methods. In the latter case, BAM! is making what we feel to be a reasonable assessment of user intent.

5.1.2 Preexisting and Nonexistent Lambdas

If BAM! finds the lambda itself does not exist in the cloud while attempting to redeploy, it will simply warn the user to run the deploy command instead. Inversely, since AWS enforces uniqueness of lambda names for a specific region and account, we designed BAM! to warn a user to redeploy whenever they attempt to deploy a lambda with the same name as a preexisting function in the cloud.

Further, considering user interaction with AWS beyond the confines of BAM!, we had to account for the possibility that developers may wish to use our framework to pull down the code for existing lambdas created without using BAM!, or for which they have no local Node.js file. To provide functionality in this regard, we added the get command to our framework. The get command combined with redeploy can be especially helpful if the lambda code contains npm dependencies. In many such cases, multiple dependencies will result in a lambda that is too big to edit in the AWS console, which means it must be edited locally and pushed back up to AWS.

5.1.3 Deleted Roles

On a related note, unless the user dictates otherwise, BAM! will create a default role to be assumed by all deployed lambdas in the absence of a specified alternative.

The associated challenge is that, while BAM! will not itself permit this operation, the user can, much the same as with other resources, delete the default role in the AWS console. Without some workaround, all future lambda deployments will fail, owing to the nonexistence of a resource pivotal to the framework’s operation. As in previous cases, our simple solution is for BAM! to check AWS for the existence of this role and rebuild it if necessary.

The ultimate point here is that BAM! was designed to function in a manner compatible with a developer’s broader interactions with AWS.

5.2 Persistence

Second, as alluded to earlier, there is a well known problem of persisting data with FaaS. To reiterate, cloud providers such as AWS will perform autoscaling on behalf of developers with a variety of services, and AWS Lambda is no exception. This means that as the demand for computational resources arises, AWS will spin up a new server instance within an ephemeral container to run whichever lambda is being invoked.

This is ultimately beneficial to developers in that the task of provisioning resources can be abstracted away from day-to-day operations. However, any data stored by a lambda, for instance via closure of its handler function over some mutable object, will not persist beyond the lifetime of the container and will typically be lost due to teardown after roughly 15 minutes of inactivity⁵.

5.2.1 NoSQL Storage

Unless a developer wishes to spend money keeping an idle server running, which would largely defeat the purpose of autoscaling serverless technologies anyway, they must find some alternative means of persisting data. For this purpose, we designed BAM! to support DynamoDB, Amazon’s service for NoSQL key-value and document storage. Supporting DynamoDB for optional persistence provides highly desirable functionality over even more lightweight frameworks such as Shep, while, at the same time, staying true to our intended use case of building and deploying small applications. We selected DynamoDB over other storage options, such as AWS RDS, Amazon’s relational database service, since the absence of rigid schema facilitates more agile and versatile development.

5.2.2 Cold Start

It should also be noted there is a related problem of server “cold start”, where a noticeable latency will occur the first time a user invokes a lambda, as a new server instance must be spun up before invocation can proceed. In extreme cases, this can add upwards of 10 seconds of latency compared with calling a lambda on a preexisting (i.e. “warm”) instance⁵. Since instances are destroyed due to prolonged inactivity, the simple act of “pinging” the server to keep it warm will work.

While this trick will suffice to prevent cold starts⁶, AWS Lambda works on a pay-per-usage model⁷, which means there is a financial benefit to letting otherwise idle servers die. Furthermore, we do not anticipate sustained or highly variable traffic for BAM! applications, nor does our predicted use case prioritize ultra-low latency. In the end, this led us to the opinion that some minor cold start issues were ultimately tolerable.

5.3 Timing

Finally, we encountered an interesting pair of challenges related to timing, specifically an issue with latency within Amazon’s systems as well as a challenge with working around Amazon's own rate limits.

5.3.1 Latency and Stateful Dependence

Recall that, by design, a single BAM! command actually comprises an automated sequence of a large number of individual SDK operations under the hood. While we maintain that abstracting away these sequences of operations drastically improves user experience, we consequently had to account for cases where one operation relies upon Amazon's system being in a particular state generated by a previous operation.

For example, during the sequence of steps required to deploy a lambda and set up an integrated API Gateway, BAM! makes SDK calls to deploy a lambda, deploy an API gateway, and subsequently give permission for the API to invoke said lambda. This may sound obvious, but in order to attach these permissions the relevant resources actually have to exist!

Similarly, in the case where BAM! must rebuild its default role before deploying a lambda, AWS must be in a ready state for the deployed lambda to assume the newly created role. Due to automation of this process, the assumption of the role will be attempted mere milliseconds later.

Within the BAM! framework’s Node.js source code, each SDK operation is performed sequentially, with each operation asynchronously awaiting its predecessor. The problem is that with the AWS SDK, if a request is made, say to deploy a lambda, Amazon comes back with an optimistic AWS Request Object⁸ in the event of success. In other words, even though we are awaiting each SDK call, the next asynchronous operation is performed upon receipt of an optimistic successful response, not when the state of Amazon's system is actually prepared to accept the next operation. This latency, if left unhandled, would periodically cause the failure of one, and therefore all subsequent SDK operations.

5.3.2 AWS Rate Limits

Similar to the challenge posed by unintended latency was that of SDK operations failing due to Amazon's own rate limits. In situations like this, Amazon throttles one of the many individual steps part way through a sequence of operations.

AWS sets a variety of per account rate limits for different interactions with its services, ranging from a comparatively lenient 5 requests per second for API Gateway resource creation to a staggering 1 request per 30 seconds for API deletion⁹. In the latter case, even though no single BAM! command will delete more than one API, even manual use can easily collide with this upper bound.

One final consideration was that all of these problems can be exacerbated by a user’s geographical proximity to the data center with which BAM! is attempting to interact, since a shorter round trip results in less time between potentially prematurely attempted SDK operations.

5.3.3 bamBam

Since both Amazon's rate limits and infrastructural latency are beyond our control, it is fair to say that this was not so much a traditional system design problem, but rather an opportunity to take a preexisting architecture and find clever workarounds to accomplish our engineering goals.

Fundamentally, both of these challenges boiled down to timing, with BAM! unceremoniously attempting operations doomed to cascading failure. The only key difference was that in the case of rate limits the relevant timing constraints were deliberately imposed.

Our solution was to gracefully retry SDK operations behind the scenes until the state of Amazon’s system is prepared to handle that operation.

To this end, we created a function, affectionately referred to as bamBam, to provide this retry functionality.

const bamBam = async (asyncFunc, {
  asyncFuncParams = [],
  retryError = 'InvalidParameterValueException',
  interval = 3000,
  retryCounter = 0,
} = {}) => {
  const withIncrementedCounter = () => (
    {
      asyncFuncParams,
      retryError,
      interval,
      retryCounter: retryCounter + 1,
    }
  );

  const retry = async (errCode) => {
    if (firstTooManyRequestsException(errCode, retryCounter)) {
      logAwsDelayMsg();
    }

    const optionalParamsObj = withIncrementedCounter();
    await delay(interval);
    const data = await bamBam(asyncFunc, optionalParamsObj);
    return data;
  };

  try {
    const data = await asyncFunc(...asyncFuncParams);
    return data;
  } catch (err) {
    const errorCode = err.code;
    if (errorCode === retryError) {
      const data = await retry(errorCode);
      return data;
    }

    throw (err);
  }
};
          

bamBam was designed to anticipate the type of exceptions that are raised upon encountering one of the two problems described above; respectively, we found the SDK throws an InvalidParameterValueException in the case of the aforementioned latency issue, and a TooManyRequestsException when AWS is throttling requests.

In either case, bamBam responds to the anticipated error by retrying a supplied async callback wrapping an SDK operation, and does so recursively in a loop until either the operation succeeds or an unanticipated exception is raised.

The reason for retrying operations only in the event of a particular error is because if a wholly different type of exception is thrown, it is preferable to simply let the operation fail rather than keep the running process hanging indefinitely, knowing full well it may never succeed. BAM! will therefore only retry operations in cases where eventual success is a likely outcome.

All of the challenges described above presented an opportunity to refine BAM! and further optimize it for building and deploying small applications. Designing a framework atop Amazon’s cloud computing services and simplifying developer interaction therewith has required a significant degree of ingenuity and will hopefully save developers a great deal of frustration.

6 Future Work

Of course, a software application is never truly finished; below are three ideas to further improve the developer’s experience with AWS via the BAM! framework.

6.1 CORS

We would like to add Cross-Origin Resource Sharing (CORS) in order to allow endpoints to be called from another application. Currently, it is easy to enable CORS for an endpoint from within the AWS web console, but our hope is for most developers to be able to use BAM! without needing to interact with the AWS console.

6.2 AWS S3

AWS S3 is Amazon’s Simple Storage Service. Since we anticipate BAM! being used often to deploy webpages, we would like to add support for uploading assets to an S3 bucket for use within lambda code.

6.3 Additional Runtimes

With the addition of the Ruby¹⁰ and custom¹¹ runtimes on Nov. 29th 2018, AWS Lambda now supports:

• Node.js
• Python
• Java
• Go
• .NET (C#, PowerShell)
• Ruby
• custom runtimes (C++, Rust)

We would like to add support for one or more of the above runtimes in addition to Node.js.