Cant connect dynamo Db from my vpc configured lambda function

Question

i need to connect elastic cache and dynamo db from a single lambda function. My code is

exports.handler = (event, context, callback) => {

    var redis = require("redis");
    var client;
    function connectRedisClient() {
        client = redis.createClient(6379, "dgdfgdfgdfgdfgdfgfd.use1.cache.amazonaws.com", { no_ready_check: true });
    }

    connectRedisClient();
    client.set('sampleKey', 'Hello World', redis.print);
    console.log("set worked");
    client.quit();


    var AWS = require("aws-sdk");
    var docClient = new AWS.DynamoDB.DocumentClient();
    var table = "dummy";
    var year = 2015;
    var title = "The Big New Movie";
    var params = {
        TableName: table,
        Item: {
            "userid": "manafcj",
            "year": year,
            "title": title,
            "test1": [645645, 7988],
            "info": {
                "plot": "Nothing happens at all.",
                "rating": 0
            }
        }
    };

    console.log("Adding a new item...");
    docClient.put(params, function (err, data) {
        if (err) {
            console.error("Unable to add item. Error JSON:", JSON.stringify(err, null, 2));
        } else {
            console.log("Added item:", JSON.stringify(data, null, 2));
        }
    });
    callback(null, 'Hello from Lambda');
 }; 

I executed this lambda code without configuring vpc, elastic cache section is not working , but dynamo insertion is done perfectly.

after that i made setup for VPC in my account by following steps.

1. create vpc name : test-vpc-name CIDR block:172.31.0.0/16 Tenancy:Default

2. Create a new subnet. name tag : test-subnet-1a CIDR block :172.31.0.0/20

   name tag : test-subnet-1b CIDR block :172.31.16.0/20

3. Create a route table name tag : test-route-table

4. Create a internet gateway name:test-internet-gateway

5. Attach VPC

6. Route all outbound 0.0.0.0/0 traffic in routes

7. Create a route table subnet association

8. Create a NAT Gateway subnet : test-subnet-1a

also i have configured my elastic cache setup by following steps

1. Create subnet cache group name : test-cache-group

2. Create elastic cache
   type: redis Cluster Name : test-cache

   subnet cache group : test-cache-group

Finally, i have configured newly created vpc on my lambda function. Then redis-elastic cache connection is working fine, but dynamo db connection is lost. I need both working fine from a single lambda function.

I think, some fault in VPC configuration with NAT Gateway.

What is the actual issue in this setup?

Answer 1

Lambda and DynamoDB are executed in the AWS Public Cloud. Both are services executed in a internet facing environment. The Elastic Cache Cluster, otherwise, is user managed service that runs on your own VPC.

The first option to give access to your elastic cache cluster to your lambda function is using a NAT instance to foward external network connections to Elastic Cache cluster inside your VPC. You can get use the instructions from this document to help you with this task.

The second option, is the one that you already tried. Amazon says that when you configure this option it does not means that the Lambda will be executed inside your VPC. What is does it define the Elastic Network Interface of the Lambda container to access your VPC. At the end of day I don't think that this makes difference. You can see the details here.

But the point is, the container where your lambda is executed has only one Elastic Network Interface. If you configure your lambda to use your VPC, the Network Interface will be configured to access your subnet using a private IP and lost the internet connection. So, it will not be able to access DynamoDB unless you have a configure NAT instance/Gateway in your VPC.

As per you told us. You configured your VPC with a NAT Gateway. If all were correctly configured, this should be working. Maybe you can try the fist option, leaving your lambda outside your VPC and configuring the NAT Gateway to route the inboud connections to your Elastic Cache Cluster.

Why don't try and tell us the result?

Answer 2

There is now a relatively easy solution: VPC Endpoints.

"Previously, if you wanted your EC2 (elroy: or lambda) instances in your VPC to be able to access DynamoDB, you had two options. You could use an Internet Gateway (with a NAT Gateway or assigning your instances public IPs) or you could route all of your traffic to your local infrastructure via VPN or AWS Direct Connect and then back to DynamoDB."

"A VPC endpoint for DynamoDB enables Amazon EC2 instances in your VPC to use their private IP addresses to access DynamoDB with no exposure to the public Internet...Your EC2 instances do not require public IP addresses, and you do not need an Internet gateway, a NAT device, or a virtual private gateway in your VPC. You use endpoint policies to control access to DynamoDB. Traffic between your VPC and the AWS service does not leave the Amazon network. "

The above quotes come from the links below. Note the the references to "EC2 instances" apply to lambda contexts as well.

See https://aws.amazon.com/blogs/aws/new-vpc-endpoints-for-dynamodb/

and

http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/vpc-endpoints-dynamodb.html

Edited to provide more details in line.