Using S3 and DynamoDB from Lambda Over a Private Network

Introduction

AWS Lambda can interact with other AWS services, such as Amazon S3 and Amazon DynamoDB, securely over a private network using AWS PrivateLink and VPC configurations. Setting up Lambda functions to access S3 and DynamoDB in a private network offers better security and control over network traffic, helping avoid exposure to the public internet. This article provides step-by-step instructions for setting up S3 and DynamoDB access from Lambda within a private network, along with the advantages of this setup.

Tip: S3 Bucket Name Generator - Use this tool to generate unique and compliant S3 bucket names.

Advantages of Private Network Access for Lambda

1. Enhanced Security: By routing traffic within AWS’s private network, you prevent data from traversing the public internet, reducing exposure to security risks.
2. Better Compliance: Private connections help meet compliance requirements, as sensitive data is not exposed to the public network.
3. Reduced Latency: Network traffic within a Virtual Private Cloud (VPC) can benefit from lower latency and optimized routing within AWS.
4. Controlled Access: Private network setups offer more granular control over network access, allowing you to apply specific VPC policies and security groups to manage access to resources.

Step-by-Step Setup Guide

Step 1: Configure a VPC and Private Subnets

1. Create a VPC:

   • Go to the VPC Console and create a new VPC with the required CIDR block, for example, 10.0.0.0/16.

2. Create Private Subnets:

   • In the VPC, create two private subnets in different availability zones, which will allow Lambda functions to run in a private network.

3. Add a NAT Gateway (Optional):

   • If Lambda functions require internet access for other purposes, you can add a NAT Gateway to a public subnet to route outbound internet traffic.

Step 2: Set Up VPC Endpoints for S3 and DynamoDB

AWS VPC endpoints allow Lambda functions within a VPC to access AWS services directly without using a public IP.

1. Create an S3 VPC Endpoint:

   • Go to the VPC Console > Endpoints > Create Endpoint.
   • Choose the S3 service and select the VPC and private subnets created in Step 1.
   • Attach the necessary security group rules to allow Lambda access.

2. Create a DynamoDB VPC Endpoint:

   • In the same VPC Console > Endpoints > Create Endpoint.
   • Choose the DynamoDB service and select the VPC and private subnets.
   • Attach security group rules to allow access only to authorized Lambda functions.

Step 3: Configure Security Groups

1. Create a Security Group:
   • Create a security group to control access between Lambda, S3, and DynamoDB.
   • Allow inbound and outbound traffic for your VPC CIDR range or restrict it to the private subnets to ensure controlled access.

Step 4: Set Up AWS Lambda with VPC Access

1. Create or Update Lambda Function:

   • Go to the Lambda Console and either create a new Lambda function or modify an existing one.

2. Configure VPC Access:

   • In the Lambda function’s configuration settings, select VPC under Network.
   • Choose the VPC and the private subnets created in Step 1, and assign the Lambda function to the security group configured in Step 3.
   • This will allow the Lambda function to run in a private subnet with access to S3 and DynamoDB through the VPC endpoints.

Step 5: Test Access to S3 and DynamoDB from Lambda

To confirm that Lambda can access S3 and DynamoDB within the private network:

1. Create Test Code: Write a simple Lambda function that reads from and writes to both S3 and DynamoDB.

   import boto3

   s3 = boto3.client('s3')
   dynamodb = boto3.client('dynamodb')

   def lambda_handler(event, context):
       # Access S3 bucket
       response = s3.get_object(Bucket='your-private-bucket', Key='example.txt')
       content = response['Body'].read().decode('utf-8')
       
       # Write data to DynamoDB
       dynamodb.put_item(
           TableName='your-table',
           Item={'id': {'S': '123'}, 'content': {'S': content}}
       )
       return {
           'statusCode': 200,
           'body': 'Data retrieved and stored successfully'
       }

2. Run a Test: Invoke the Lambda function in the Lambda Console or via CLI and check the S3 bucket and DynamoDB table for successful read and write operations.

Step 6: Verify Private Connectivity

1. Monitor in VPC Flow Logs:

   • Enable VPC flow logs to verify that traffic is routed through VPC endpoints and remains within AWS’s private network.

2. Check CloudWatch Logs:

   • Ensure that your Lambda function’s logs indicate successful access to S3 and DynamoDB over the private network.

FAQ: Can This Lambda Be Invoked from the Public Internet?

No, if the Lambda function is configured within a VPC without additional public access configurations, it won’t be directly accessible from the public internet. By default, placing a Lambda function in a private subnet within a VPC isolates it from public access.

However, there are ways to allow external access while keeping the Lambda function’s operations private:

1. API Gateway: You can use API Gateway as an entry point, which securely invokes the Lambda function. This allows public internet requests to reach API Gateway, but the Lambda function itself remains isolated in the private VPC.

2. Lambda Function URLs: AWS Lambda Function URLs provide a direct HTTP(S) endpoint to invoke a Lambda function. When used with VPC, additional security controls and IAM policies would be needed to ensure only authorized requests can access it.

3. Application Load Balancer (ALB): An ALB can also route traffic from the public internet to Lambda functions, offering more flexibility for security configurations and allowing you to set access restrictions via ALB rules.

By using one of these services, you can allow public access to the Lambda function securely, without exposing the Lambda function or its VPC configuration to the internet directly.

Conclusion

By configuring Lambda functions to access S3 and DynamoDB over a private network, you can enhance security, reduce latency, and better control access to sensitive resources. This setup helps protect your data by keeping it within AWS’s private network, avoiding exposure to the public internet. Following the above steps will allow you to set up a secure and efficient environment for your Lambda functions, ensuring they can communicate with other AWS services while maintaining compliance and optimizing performance.