Bert Word Alignment Sagemaker model

.BERT_SAGEMAKER.md

hello

.gitignore

*.pyc
__pycache__
env
tmp

delete_stack.sh

#!/usr/bin/env bash

aws cloudformation delete-stack --stack-name $STACK_NAME

deploy_stack.sh

#!/usr/bin/env bash

aws cloudformation deploy --stack-name $STACK_NAME --template-file stack.yaml --capabilities CAPABILITY_NAMED_IAM  --parameter-overrides RepoName=$REPO_NAME

list_events.sh

#!/usr/bin/env bash

aws cloudformation describe-stack-events --stack-name $STACK_NAME --query "StackEvents[].{\"1.Timestamp\":Timestamp,\"2.Id\":LogicalResourceId,\"3.Type\":ResourceType,\"4.Status\":ResourceStatus,\"5.Reason\":ResourceStatusReason}"

list_images.sh

#!/usr/bin/env bash

aws ecr list-images --repository-name "$REPO_NAME" --query 'imageIds[*]' --output table

list_outputs.sh

#!/usr/bin/env bash

aws cloudformation describe-stacks --stack-name $STACK_NAME --query 'Stacks[0].Outputs' --output table

list_resources.sh

#!/usr/bin/env bash

aws cloudformation describe-stack-resources --stack-name $STACK_NAME --query "StackResources[].{\"1.Timestamp\":Timestamp,\"2.LogicalId\":LogicalResourceId,\"3.PhysicalId\":PhysicalResourceId,\"4.Type\":ResourceType,\"5.Status\":ResourceStatus}"

list_stacks.sh

#!/usr/bin/env bash

aws cloudformation describe-stacks --query "Stacks[].{\"1.Name\":StackName,\"2.Status\":StackStatus}"

NOTES.md

logs 19/05/24

• sagemaker -> inference -> endpoints -> [click endpoint name] -> logs

ping_app.py

import boto3, json, os, sys

def fetch_outputs(cf, stackname):
    outputs = {}
    for stack in cf.describe_stacks()["Stacks"]:
        if (stack["StackName"].startswith(stackname) and
            "Outputs" in stack):
            for output in stack["Outputs"]:
                outputs[output["OutputKey"]] = output["OutputValue"]
    return outputs

def translate_text(text,
                   source_language_code = "ru",
                   target_language_code = "en"):
    translate = boto3.client('translate')
    result = translate.translate_text(
        Text=text,
        SourceLanguageCode=source_language_code,
        TargetLanguageCode=target_language_code
    )
    return result['TranslatedText']

def columnise(text, n = 48):
    return text[:n] if len(text) > n else text+" ".join(["" for i in range(n-len(text))])

if __name__ == "__main__":
    try:
        if "STACK_NAME" not in os.environ:
            raise RuntimeError("STACK_NAME not defined")
        stack_name = os.environ["STACK_NAME"]
        if len(sys.argv) < 2:
            raise RuntimeError("please enter Russian text")
        src = sys.argv[1]
        tgt = translate_text(src)
        struct = {
            "src": src,
            "tgt": tgt,
        }
        cf, sagemaker = (boto3.client("cloudformation"),
                         boto3.client('sagemaker-runtime'))
        outputs = fetch_outputs(cf, stack_name)
        endpoint_name = outputs["SageMakerEndpointName"]
        response = sagemaker.invoke_endpoint(
            EndpointName = endpoint_name,
            ContentType = 'application/json',
            Accept = 'application/json',
            Body = json.dumps(struct)
        )
        pairs = json.loads(response['Body'].read().decode())
        for text, translation in pairs:
            print (f"{columnise(text)}\t{columnise(translation)}")   
    except RuntimeError as error:
        print ("Error: %s" % str(error))
        

requirements.txt

awscli
boto3
botocore

setenv.sh

#!/usr/bin/env bash

export AWS_PROFILE=woldeploy
export AWS_DEFAULT_OUTPUT=table
export AWS_REGION=eu-west-1
export AWS_ACCOUNT_ID=119552584133
export REPO_NAME=bert-word-alignment
export STACK_NAME=bwa-model-stack

stack.yaml

AWSTemplateFormatVersion: '2010-09-09'
Parameters:
  RepoName:
    Type: String
  ImageTag:
    Type: String
    Default: latest
  ModelMode:
    Type: String
    Default: SingleModel
  InstanceType:
    Type: String
    Default: 'ml.m5.large'
Resources:
  SageMakerExecutionRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: 
                - 'sagemaker.amazonaws.com'
            Action: 'sts:AssumeRole'
      Policies:
        - PolicyName: 'SageMakerFullAccessPolicy'
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - 'sagemaker:*'
                  - 'logs:*'
                  - 's3:*'
                  - 'ecr:*'
                Resource: '*'
  SageMakerModel:
    Type: 'AWS::SageMaker::Model'
    Properties: 
      ModelName: !Sub "${RepoName}-model"
      ExecutionRoleArn: !GetAtt SageMakerExecutionRole.Arn
      PrimaryContainer:
        Image: !Sub '${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${RepoName}:${ImageTag}'
        Mode: !Ref ModelMode
        Environment: {}
  SageMakerEndpointConfig:
    Type: 'AWS::SageMaker::EndpointConfig'
    Properties: 
      ProductionVariants: 
        - VariantName: 'AllTraffic'
          ModelName: !GetAtt SageMakerModel.ModelName
          InitialInstanceCount: 1
          InstanceType: !Ref InstanceType
          InitialVariantWeight: 1.0
  SageMakerEndpoint:
    Type: 'AWS::SageMaker::Endpoint'
    Properties: 
      EndpointConfigName: !GetAtt SageMakerEndpointConfig.EndpointConfigName
Outputs:
  SageMakerModelName:
    Value: !GetAtt SageMakerModel.ModelName
  SageMakerEndpointConfigName:
    Value: !GetAtt SageMakerEndpointConfig.EndpointConfigName
  SageMakerEndpointName:
    Value: !GetAtt SageMakerEndpoint.EndpointName

TODO.md

short

done

• change stack to output names
• ping_app to lookup endpoint name
• test removing depends
• test removing hardcoded names
• direct integration/invocation test
• web url test
• check model/endpoint still deploys

 (rapid) exec '/var/runtime/bootstrap' (cwd=/app, handler=)

• list images
• deploy stack
• delete stack