Skip to content

Instantly share code, notes, and snippets.

@srcecde
Created June 27, 2018 09:11
Show Gist options
  • Save srcecde/f4d992a7f9765e391ee938b6176614de to your computer and use it in GitHub Desktop.
Save srcecde/f4d992a7f9765e391ee938b6176614de to your computer and use it in GitHub Desktop.
AWS: Lambda with Comprehend for Batch processing
import boto3
def datachunk(para):
text_list = []
while para:
text_list.append(str(para[:4700]))
para = para[4700:]
return text_list[:25]
def lambda_handler(event, context):
s3 = boto3.client("s3")
bucket = "bucket-name"
key = "filename.txt"
file = s3.get_object(Bucket = bucket, Key = key)
paragraph = str(file["Body"].read().decode("utf-8"))
comprehend = boto3.client("comprehend")
response = comprehend.batch_detect_key_phrases(TextList = datachunk(paragraph), LanguageCode = "en")
for i in response["ResultList"]:
for j in i["KeyPhrases"] :
print(j["Text"])
return 'Thanks for using'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment