Skip to content

Instantly share code, notes, and snippets.

@heitorlessa
Last active December 19, 2021 10:25
Show Gist options
  • Save heitorlessa/07cae5b08ffde85fe69e03ed1dbc5c88 to your computer and use it in GitHub Desktop.
Save heitorlessa/07cae5b08ffde85fe69e03ed1dbc5c88 to your computer and use it in GitHub Desktop.
SAM support for new SQS, Kinesis, and DynamoDB Partial Response feature (ReportBatchItemFailures)
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: partial batch response sample
Globals:
Function:
Timeout: 5
MemorySize: 256
Runtime: python3.8
Tracing: Active
Environment:
Variables:
# Powertools env vars: https://awslabs.github.io/aws-lambda-powertools-python/#environment-variables
LOG_LEVEL: INFO
POWERTOOLS_SERVICE_NAME: hello
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function
Properties:
Handler: app.lambda_handler
CodeUri: hello_world
Policies:
- SQSPollerPolicy:
QueueName: !GetAtt SampleQueue.QueueName
# Lambda Destinations require additional permissions
# to send failure records from Kinesis/DynamoDB
- Version: "2012-10-17"
Statement:
Effect: "Allow"
Action:
- sqs:GetQueueAttributes
- sqs:GetQueueUrl
- sqs:SendMessage
Resource: !GetAtt SampleDLQ.Arn
Events:
Batch:
Type: SQS
Properties:
Queue: !GetAtt SampleQueue.Arn
FunctionResponseTypes:
- ReportBatchItemFailures
KinesisStream:
Type: Kinesis
Properties:
Stream: !GetAtt SampleStream.Arn
BatchSize: 100
StartingPosition: LATEST
MaximumRetryAttempts: 2
DestinationConfig:
OnFailure:
Destination: !GetAtt SampleDLQ.Arn
FunctionResponseTypes:
- ReportBatchItemFailures
DynamoDBStream:
Type: DynamoDB
Properties:
Stream: !GetAtt SampleTable.StreamArn
StartingPosition: LATEST
MaximumRetryAttempts: 2
DestinationConfig:
OnFailure:
Destination: !GetAtt SampleDLQ.Arn
FunctionResponseTypes:
- ReportBatchItemFailures
SampleDLQ:
Type: AWS::SQS::Queue
SampleQueue:
Type: AWS::SQS::Queue
Properties:
VisibilityTimeout: 30 # Fn timeout * 6
RedrivePolicy:
maxReceiveCount: 2
deadLetterTargetArn: !GetAtt SampleDLQ.Arn
SampleStream:
Type: AWS::Kinesis::Stream
Properties:
ShardCount: 1
SampleTable:
Type: AWS::DynamoDB::Table
Properties:
BillingMode: PAY_PER_REQUEST
AttributeDefinitions:
- AttributeName: pk
AttributeType: S
- AttributeName: sk
AttributeType: S
KeySchema:
- AttributeName: pk
KeyType: HASH
- AttributeName: sk
KeyType: RANGE
SSESpecification:
SSEEnabled: yes
StreamSpecification:
StreamViewType: NEW_IMAGE
AWSTemplateFormatVersion: '2010-09-09'
Description: partial batch response sample
Resources:
SampleDLQ:
Type: AWS::SQS::Queue
SampleQueue:
Type: AWS::SQS::Queue
Properties:
VisibilityTimeout: 30
RedrivePolicy:
maxReceiveCount: 2
deadLetterTargetArn:
Fn::GetAtt:
- SampleDLQ
- Arn
SampleStream:
Type: AWS::Kinesis::Stream
Properties:
ShardCount: 1
SampleTable:
Type: AWS::DynamoDB::Table
Properties:
BillingMode: PAY_PER_REQUEST
AttributeDefinitions:
- AttributeName: pk
AttributeType: S
- AttributeName: sk
AttributeType: S
KeySchema:
- AttributeName: pk
KeyType: HASH
- AttributeName: sk
KeyType: RANGE
SSESpecification:
SSEEnabled: true
StreamSpecification:
StreamViewType: NEW_IMAGE
HelloWorldFunction:
Properties:
Code:
S3Bucket: bucket
S3Key: value
Environment:
Variables:
LOG_LEVEL: INFO
POWERTOOLS_SERVICE_NAME: hello
Handler: app.lambda_handler
MemorySize: 256
Role:
Fn::GetAtt:
- HelloWorldFunctionRole
- Arn
Runtime: python3.8
Tags:
- Key: lambda:createdBy
Value: SAM
Timeout: 5
TracingConfig:
Mode: Active
Type: AWS::Lambda::Function
HelloWorldFunctionRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Version: '2012-10-17'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
- arn:aws:iam::aws:policy/AWSXrayWriteOnlyAccess
- arn:aws:iam::aws:policy/service-role/AWSLambdaSQSQueueExecutionRole
- arn:aws:iam::aws:policy/service-role/AWSLambdaKinesisExecutionRole
- arn:aws:iam::aws:policy/service-role/AWSLambdaDynamoDBExecutionRole
Policies:
- PolicyDocument:
Statement:
- Action:
- sqs:ChangeMessageVisibility
- sqs:ChangeMessageVisibilityBatch
- sqs:DeleteMessage
- sqs:DeleteMessageBatch
- sqs:GetQueueAttributes
- sqs:ReceiveMessage
Effect: Allow
Resource:
Fn::Sub:
- arn:${AWS::Partition}:sqs:${AWS::Region}:${AWS::AccountId}:${queueName}
- queueName:
Fn::GetAtt:
- SampleQueue
- QueueName
PolicyName: HelloWorldFunctionRolePolicy0
- PolicyDocument:
Version: '2012-10-17'
Statement:
Effect: Allow
Action:
- sqs:GetQueueAttributes
- sqs:GetQueueUrl
- sqs:SendMessage
Resource:
Fn::GetAtt:
- SampleDLQ
- Arn
PolicyName: HelloWorldFunctionRolePolicy1
Tags:
- Key: lambda:createdBy
Value: SAM
Type: AWS::IAM::Role
HelloWorldFunctionBatch:
Properties:
EventSourceArn:
Fn::GetAtt:
- SampleQueue
- Arn
FunctionName:
Ref: HelloWorldFunction
FunctionResponseTypes:
- ReportBatchItemFailures
Type: AWS::Lambda::EventSourceMapping
HelloWorldFunctionKinesisStream:
Properties:
BatchSize: 100
DestinationConfig:
OnFailure:
Destination:
Fn::GetAtt:
- SampleDLQ
- Arn
EventSourceArn:
Fn::GetAtt:
- SampleStream
- Arn
FunctionName:
Ref: HelloWorldFunction
FunctionResponseTypes:
- ReportBatchItemFailures
MaximumRetryAttempts: 2
StartingPosition: LATEST
Type: AWS::Lambda::EventSourceMapping
HelloWorldFunctionStream:
Properties:
DestinationConfig:
OnFailure:
Destination:
Fn::GetAtt:
- SampleDLQ
- Arn
EventSourceArn:
Fn::GetAtt:
- SampleTable
- StreamArn
FunctionName:
Ref: HelloWorldFunction
FunctionResponseTypes:
- ReportBatchItemFailures
MaximumRetryAttempts: 2
StartingPosition: LATEST
Type: AWS::Lambda::EventSourceMapping
Outputs:
HelloWorldFunction:
Value:
Fn::GetAtt:
- HelloWorldFunction
- Arn
Queue:
Value:
Ref: SampleQueue
Stream:
Value:
Ref: SampleStream
Table:
Value:
Ref: SampleTable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment