Change Python logger format in AWS Lambda

change_lambda_logger_format.py

# Python logger in AWS Lambda has a preset format. To change the format of the logging statement, 
# remove the logging handler & add a new handler with the required format

import logging
import sys

def setup_logging():
    logger = logging.getLogger()
    for h in logger.handlers:
      logger.removeHandler(h)
    
    h = logging.StreamHandler(sys.stdout)
    
    # use whatever format you want here
    FORMAT = '%(asctime)s %(message)s'
    h.setFormatter(logging.Formatter(FORMAT))
    logger.addHandler(h)
    logger.setLevel(logging.INFO)
    
    return logger

def lambda_handler(event, context):
    logger = setup_logging()
    logger.info("This is a test log statement!")
    return
    
# Expected output from Lambda:
# 
# START RequestId: 1a2b3c4d-abcd-1234-efgh-1a2b3c4d5e6f Version: $LATEST
# 2017-10-06 22:40:59,653 This is a test log statement!
# END RequestId: 1a2b3c4d-abcd-1234-efgh-1a2b3c4d5e6f
# REPORT RequestId: 1a2b3c4d-abcd-1234-efgh-1a2b3c4d5e6f	Duration: 0.41 ms	Billed Duration: 100 ms 	Memory Size: 128 MB	Max Memory Used: 21 MB

I was able to retain aws_request_id in the log output:
https://gist.github.com/matthewpick/3aa01abfeda36eae717837a99994d3ed

has anyone managed to find a nice way to get "aws_request_id" when using the approach of removing the default handler and using your own?

Since the default handler's filter injects the aws_request_id into the record, if you remove the default handler, you will not be able to retrieve aws_request_id. Therefore, instead of removing the default handler and adding your own, you should just modify the formatter using setFormatter.

Also you can see the code of default handler and filter
https://github.com/aws/aws-lambda-python-runtime-interface-client/blob/849e874de01776cb386c18fb2c1731d43cd2b2f4/awslambdaric/bootstrap.py#L339C1-L342C20