API Gateway is the best companion of Lambda, and many people try their first Lambda function with API Gateway. But what many people don’t know is that API Gateway is not just a trigger, it can do more than you may think.

This article will walk you through on how to use API Gateway to:

Validate user input
Transform data format
Integrate with other AWS services

1. Input validation

One of the most crucial security principles is never trusting user input. So you probably have some code validating user input at the beginning of the Lambda function.

Instead of invoking the Lambda function to validate user input and return errors, we can validate the input using API Gateway in the first place and avoid unnecessary Lambda invocation. Here’s how we can do that:

Step 1: Creating the data model

First, we need to define the model of our desired user input so that API Gateway can use it to do the validation.

In AWS console, after you create a REST API, you can see a page called Models.

In the Model page, we can create data models using JSON Schema — In the **Model** page, we can create data models using JSON Schema

Model is a JSON schema, which you can define the expected structure of request content. For example, I can define my API endpoint to ingest a JSON containing a stringname ; an integer id ; and no more extra value, by the following schema:

{
    "$schema":"http://json-schema.org/draft-04/schema#",
    "properties":{
        "name":{
            "type":"string"
        },
        "id":{
            "type":"integer"
        }
    },
    "required":[
        "id",
        "name"
    ],
    "additionalProperties":false
}

It’s just a simple example. With JSON Schema, you can define more complex validation rules, like min/max values, enum, pattern matching, nested object.

Step 2: Binding model with the API endpoint

Next, we need to bind the model to the endpoint we want to do the validation. In the Resouces page, click into the endpoint and choose Method Request.

In Method Request page, we can bind the model to an endpoint — In **Method Request** page, we can bind the model to an endpoint

We choose “Validate body, query string parameters, and headers” for Request Validator so that API Gateway validates the request body from the user.

Under the Request Body tab, we add a row with Content Type set to application/json (of course you can use other content types if your API receive different content types). In the Model name column, we choose the model we have created in step 1.

Now, we can test our API. Go back to the Method Execution page and click Test. On this page, we can preview the behaviour of the API without deploying it.

The API returns an HTTP 400 error if the input is not valid

I try using an empty object as input. Because the model states that name and id are required, the API immediately returns an HTTP 400 error without invoking downstream processes.

Step 3: Modify the error response

As we can see, the API returns an error message “Invalid request body”. If we input a JSON with name and id but in the wrong data type, for example:

{
    "name": "Richard",
    "id": "This is a string"
}

It still returns the same message. It’s not very useful as it doesn’t tell the user which part of the request is invalid.

In Gateway Responses page, we can customise the response for each response type. For our use case, we are going to customise Bad Request Body.

We can customise the response from API Gateway

Click on application/json under Response templates, we can see the default template is:

{"message":$context.error.messageString}

We need to change it to a more meaningful message. There are many variables we can use to construct the response body. In our case, we are going to use $context.error.validationErrorString , we change the template into:

{"message":"$context.error.validationErrorString"}

Using $context variable to contract the response body

After saving the template, we can go back to Method Execution page and do the testing again.

Now, if we input an empty object as input, API Gateway responses an HTTP 400 error with the message

[object has missing required properties (["id","name"])]

If we put a string in id field, it responses with the message

[instance type (string) does not match any allowed primitive type (allowed: ["integer"])]

The response contains the validation error now

2. Data Transformation and Service Integration

Many times when people want to provide an AWS service to customers, they don’t directly expose the service API simply because they want to control the request and response format. The first thing in mind must be using Lambda to transform the data format and make the API call to the AWS service.

Example: Getting record from DynamoDB

Querying record from DynamoDB should be one of the most common API use cases. API Gateway can help us translate query string from user input into DynamoDB query. And translate the result into our desire format to the user.

In this example, I am going to create a GET endpoint, taking id from the query string, then return the related record from DynamoDB.

Step 1: Create an IAM role

We need to create an IAM role for API Gateway to access AWS services. In this example, we are going to create a role granting DynamoDB Query permission.

In the IAM Roles page, we create a new role. For “trusted entity”, we choose API Gateway. Then, we leave all config as the default until the last step. Give the role a name, e.g. DynamoDBQueryForAPIGateway.

Next, we need to detach the default AmazonAPIGatewayPushToCloudWatchLogs policy and add our inline policy.

Remove default policy and add our inline policy

In the inline policy, we grant DynamoDB Query permission on the table.

Add the inline policy to grant query access on DynamoDB table

Step 2: Require query string parameters from users

Under the Method Request page of the GET method, we choose “Validate query string parameters and headers” for Request Validator.

Under the “URL Query String Parameters” section, we add a row with name id and mark as required.

Now, the endpoint checks if the user provides parameter id in the query string, and reject the call if it does not exist.

Make id as a required parameter of the endpoint.

Step 3: Transform user input into DynamoDB API call

Go to Integration Request page, that’s the place we construct the DynamoDB call.

Choose AWS Service for Integration type; DynamoDB for AWS Service; and choose the region of the DynamoDB table for AWS Region.

For execution role, put the ARN of the role we have created in step 1.

For HTTP method and Action, we can check out the API reference. Each AWS service has its API request format. In this case, we are going to use POST Query (API Reference).

Scroll down to the bottom, and we can see the Mapping Templates section, that’s is the place we construct the API call body. — Scroll down to the bottom, and we can see the **Mapping Templates** section, that’s is the place we construct the API call body.

We are going to add a new mapping template, with Content-Type set to application/json (We are expecting JSON from user requests).

For Request body passthrough, we choose Never. Any request which is not a JSON will be rejected so that outsiders cannot bypass our mapping and directly access the DynamoDB API.

Using mapping template to construct DynamoDB API call

The most exciting part comes, the template itself. The mapping template in API Gateway uses Apache Velocity Template Language (VTL) format. We can use VTL to construct the API call body, just as we are constructing API call as usual. Combining with some simple logic, like if-condition, for-loop, and the variables provided by API Gateway, we can make DynamoDB API call based on the user input.

In this example, we are going to construct a usual Query call, using id parameter provided by the user as the primary key condition:

{
    "TableName": "xxxxxxxx",
    "KeyConditionExpression": "id = :id",
    "ExpressionAttributeValues": {
        ":id": {
            "N": "$input.params('id')"
        }
    }
}

Step 4: Transform query result to the API response

Similar to constructing API call, we can construct the response to our users based on the result from DynamoDB.

In Integration Response page, we can configure the mapping between the response from DynamoDB and the response sending to users. To make it simple, we are going to use the default one.

Under the Mapping Templates section, we add a new template with Content-Type set to application/json.

Using mapping template to convert DynamoDB response into user response

For the template body, we are going to use a for-loop to list out all the items queried, and show them as a simple JSON object instead of the original format provided by DyanmoDB.

Alternatively, we can use an if condition to check if there is any record returned. We can even override the HTTP response code to 404 if there is no record returned. (AWS documentation of overriding status code)

#set($inputRoot = $input.path('$'))#if($inputRoot.Items.size() == 0)
#set($context.responseOverride.status = 404)
{
    "message":"Record not found"
}
#else
[
#foreach($item in $inputRoot.Items)
    {
        "id" : $item.id.N,
        "name" : "$item.name.S"
    }#if($foreach.hasNext),#end
#end
]
#end

Try it out

Now, if we go back to the Test page, we can provide the query string and try out the API we have just created.

If we put id=1 in the query string (which I had created the record in my DynamoDB table), we can see the API returning the JSON array we have defined, instead of the complex response structure from DynamoDB.

The API returns the records in the format that we defined.

If we put id=2 (which I don’t have any record with this id), the API returns an HTTP 404 error, along with the error message we have defined in the template.

The API returns 404 error when no records found.

Wrap up

Costs

For many web applications, most of the traffic is read traffic. If we implement them in API Gateway, we can save the cost of those unnecessary Lambda invocations.

Even though the Lambda invocation is inevitable, we can still do the input validation in API Gateway. By guarding the Lambda function from silly mistakes made by the users, we can avoid wasting invocation on doing meaningless things.

Concurrency

You may notice that API Gateway gives you 5000 burst concurrency. However, if every API call ends up in a Lambda function, this concurrency becomes 1000 because Lambda only gives you this amount of concurrency.

By lifting those simple API calls (especially the read traffics) away from Lambda, we can reserve the precious Lambda concurrency to more complex tasks.

Feature image