AWS Lambda Static Type Checker Example (Python3)

aws-lambda-static-type-checker.md

How to use Python3 Type Hints in AWS Lambda

TL;DR

Static Type Checkers help you find simple (but subtle) bugs in your Python code. Check out lambda_types.py and incrementally improve your code base and development/debugging experience with type hints.

Your Lambda Function code will go from this:

def handler(event, context):
    first_name = event.get('first_name') or 'John'
    last_name = event.get('last_name') or 'Smith'
    return {
        'message': get_message(first_name, last_name),
    }

def get_message(first_name, last_name):
    return 'Hello {} {}!'.format(first_name, last_name)

to this:

def handler(event: LambdaDict, context: LambdaContext) -> LambdaDict:
    first_name: str = event.get('first_name') or 'John'
    last_name: str = event.get('last_name') or 'Smith'
    return {
        'message': get_message(first_name, last_name),
    }


def get_message(first_name: str, last_name: str):
    return 'Hello {} {}!'.format(first_name, last_name)

Check out the two Lambda Functions below for more examples.

Instructions

• Create a Python3 virtual env with virtualenv venv --python=python3 && source venv/bin/activate
• Install mypy with pip install mypy
• Run mypy YOURFILE.py or add mypy to your IDE linting configuration (for VSCode, you'll need to enable the python.linting.mypyEnabled setting)
• Create a local lambda_types.py file (you can find it below) and customize it as needed
• Learn more about the built-in typing module here
• (Just FYI: you can run all the tests below with Nose: pip install nose and then simply run nosetests)

Tips & Tricks

• Static code annotations will not affect your code execution, as they are only useful for static checks and code completion
• Of course, static typing works fine with Python classes as well
• Make sure your functions/methods are annotated if you want them to be checked
• You can use either annotations (natively supported only by Python3, sometimes bring to decreased readability) or special comments (e.g. # type: str)
• You don't have to annotate every single function/file of your codebase to benefit from static type checking (i.e. you could focus on critical or semantically ambiguous/complex sections)
• Writing tests becomes easier/faster as well since many type-related errors will be detected at "compile time" and you won't have to unit-test them (for example, see the lambda_repeat.get_output function below)
• You can customize LambdaDict based on your own event structures and conventions. For example, you may want to use Dict[str, str] or Dict[str, Dict[str, Any]] instead of Dict[str, Any]
• Thanks to code completion, you won't have to memorize all the LambdaContext attributes and methods (e.g. context.get_remaining_time_in_millis(), context.client_context.client.installation_id, etc.)

aws-lambda-static-type-checker.png

lambda_message.py

""" Example #1 """
import os
from lambda_types import LambdaDict, LambdaContext

MSG_TEMPLATE: str = os.environ.get('MSG_TEMPLATE') or 'Hello {} {}!'
STAGE: str = os.environ.get('stage') or 'dev'


def handler(event: LambdaDict, context: LambdaContext) -> LambdaDict:
    print('Received event {} for stage {}'.format(event, STAGE))
    first_name: str = event.get('first_name')  # optional
    last_name: str = event.get('last_name')  # optional
    return {
        'message': get_message(first_name, last_name),
    }


def get_message(first_name: str = 'John', last_name: str = 'Smith'):
    return MSG_TEMPLATE.format(first_name, last_name)

lambda_repeat.py

""" Example #2 """
import os
from lambda_types import LambdaDict, LambdaContext

N: int = int(os.environ.get('N') or 10)
STAGE: str = os.environ.get('stage') or 'dev'


def handler(event: LambdaDict, context: LambdaContext) -> LambdaDict:
    print('Received event {} for stage {}'.format(event, STAGE))
    input: str = event['input']  # required
    return {
        'output': get_output(input, N),
    }


def get_output(input: str, num: int):
    """ Return the input string repeated N times. """
    return input * num

lambda_types.py

""" Note: this code is used only by the static type checker! """
from typing import Dict, Any

LambdaDict = Dict[str, Any]


class LambdaCognitoIdentity(object):
    cognito_identity_id: str
    cognito_identity_pool_id: str


class LambdaClientContextMobileClient(object):
    installation_id: str
    app_title: str
    app_version_name: str
    app_version_code: str
    app_package_name: str


class LambdaClientContext(object):
    client: LambdaClientContextMobileClient
    custom: LambdaDict
    env: LambdaDict


class LambdaContext(object):
    function_name: str
    function_version: str
    invoked_function_arn: str
    memory_limit_in_mb: int
    aws_request_id: str
    log_group_name: str
    log_stream_name: str
    identity: LambdaCognitoIdentity
    client_context: LambdaClientContext

    @staticmethod
    def get_remaining_time_in_millis() -> int:
        return 0

tests.py

import unittest
from lambda_types import LambdaDict, LambdaContext
from lambda_message import handler as handler_message, get_message
from lambda_repeat import handler as handler_repeat, get_output


class TestMessageFunction(unittest.TestCase):

    def setUp(self):
        self.context = LambdaContext()

    def test_handler(self) -> None:
        event: LambdaDict = {
            "first_name": "Alex",
            "last_name": "Casalboni",
        }
        result = handler_message(event, self.context)
        self.assertIn('message', result)

    def test_handler_empty(self) -> None:
        event: LambdaDict = {}
        result = handler_message(event, self.context)
        self.assertIn('message', result)

    def test_message_default(self) -> None:
        msg = get_message()
        self.assertIsInstance(msg, str)
        self.assertIn('Hello', msg)
        self.assertIn('John', msg)
        self.assertIn('Smith', msg)
        self.assertTrue(msg.endswith('!'))

    def test_message_firstname(self) -> None:
        msg = get_message(first_name='Charlie')
        self.assertIsInstance(msg, str)
        self.assertIn('Hello', msg)
        self.assertIn('Charlie', msg)
        self.assertIn('Smith', msg)
        self.assertTrue(msg.endswith('!'))

    def test_message_lastname(self) -> None:
        msg = get_message(last_name='Brown')
        self.assertIsInstance(msg, str)
        self.assertIn('Hello', msg)
        self.assertIn('John', msg)
        self.assertIn('Brown', msg)
        self.assertTrue(msg.endswith('!'))

    def test_message(self) -> None:
        msg = get_message(first_name='Charlie', last_name='Brown')
        self.assertIsInstance(msg, str)
        self.assertIn('Hello', msg)
        self.assertIn('Charlie', msg)
        self.assertIn('Brown', msg)
        self.assertTrue(msg.endswith('!'))


class TestRepeatFunction(unittest.TestCase):

    def setUp(self):
        self.context = LambdaContext()

    def test_handler(self) -> None:
        event: LambdaDict = {
            "input": "NaN",
        }
        result = handler_repeat(event, self.context)
        self.assertIn('output', result)
        self.assertEqual(30, len(result['output']))

    def test_handler_empty(self) -> None:
        event: LambdaDict = {}
        with self.assertRaises(KeyError):
            handler_repeat(event, self.context)

    def test_repeat_empty_string(self) -> None:
        output = get_output('', 100)
        self.assertIsInstance(output, str)
        self.assertEqual(0, len(output))

    def test_repeat_zero(self) -> None:
        output = get_output('hello', 0)
        self.assertIsInstance(output, str)
        self.assertEqual(0, len(output))

    def test_repeat(self) -> None:
        output = get_output('hello', 10)
        self.assertIsInstance(output, str)
        self.assertEqual(50, len(output))

@alexcasalboni thanks for the example.
Coming from using the Serverless framework with TypeScript, I was surprised how the template generated off of the Python implementation of Serverless does not include any type information (unlike the TypeScript one).

Moreover, given there's a specification in place (see PEP-561) on how to package and distribute type information, I'm surprised there isn't one with the types above.
Perhaps we can extract the above and distribute it as the PEP describes.

Leaving a 👍 on this gist for kudos and encouragement. Came across this doing a deep dive on a Python project leveraging type hinting, AWS Lambda, and Serverless. Thank you @alexcasalboni

Great work! 👍

👌

Thanks for this. Two requests. First, and most importantly, would you please add a license? Second, have you considered putting this on PyPi?

@P-Daddy https://github.com/illagrenan/lambda-typing

This might get merged soon too (Lambda PowerTools): aws-powertools/powertools-lambda-python#149

Type hints for various AWS services that can trigger a Lambda function: https://pypi.org/project/aws-lambda-typing

Demo: