Created
August 10, 2023 11:46
-
-
Save robshox/6f7ad341d1878d8a988b2d6a0eb69bf4 to your computer and use it in GitHub Desktop.
Function Code and JSON prompt for tutorial from Rob Shocks on AWS LLAMA 2
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## LAMNDA FUNCTION lambda_function.py | |
import boto3 | |
import json | |
# grab environment variables | |
ENDPOINT_NAME = "jumpstart-dft-meta-textgeneration-llama-2-7b-rs" | |
runtime= boto3.client('runtime.sagemaker') | |
def lambda_handler(event, context): | |
response = runtime.invoke_endpoint(EndpointName=ENDPOINT_NAME, | |
ContentType='application/json', | |
Body=event['body'], | |
CustomAttributes="accept_eula=true") | |
response_content = response['Body'].read().decode() | |
result = json.loads(response_content) | |
return { | |
"statusCode": 200, | |
"body": json.dumps(result) | |
} | |
## POSTMAN JSON PAYLOAD | |
{ | |
"inputs": [ | |
[ | |
{"role": "system", "content": "You are an expert in copywriting"}, | |
{"role": "user", "content": "Write me a tweet about super conductors"} | |
] | |
], | |
"parameters": {"max_new_tokens": 256, "top_p": 0.9, "temperature": 0.6} | |
} | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I followed the tutorial closely and checked all of my code, but this is the only thing I get returned in Postman:
{"code": 424, "message": "Batch inference failed", "properties": {}, "content": {"keys": [], "values": []}, "cancelled": false}
Here is the error message from testing:
{
"errorMessage": "'body'",
"errorType": "KeyError",
"requestId": "7d147730-2007-4263-971b-77aaea8ddd76",
"stackTrace": [
" File "/var/task/lambda_function.py", line 11, in lambda_handler\n Body=event['body'],\n"
]
}
Function Logs
START RequestId: 7d147730-2007-4263-971b-77aaea8ddd76 Version: $LATEST
LAMBDA_WARNING: Unhandled exception. The most likely cause is an issue in the function code. However, in rare cases, a Lambda runtime update can cause unexpected function behavior. For functions using managed runtimes, runtime updates can be triggered by a function change, or can be applied automatically. To determine if the runtime has been updated, check the runtime version in the INIT_START log entry. If this error correlates with a change in the runtime version, you may be able to mitigate this error by temporarily rolling back to the previous runtime version. For more information, see https://docs.aws.amazon.com/lambda/latest/dg/runtimes-update.html
[ERROR] KeyError: 'body'
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 11, in lambda_handler
Body=event['body'],END RequestId: 7d147730-2007-4263-971b-77aaea8ddd76
REPORT RequestId: 7d147730-2007-4263-971b-77aaea8ddd76 Duration: 10.39 ms Billed Duration: 11 ms Memory Size: 128 MB Max Memory Used: 74 MB Init Duration: 396.95 ms