AWS Gen AI (public beta)

cheatsheet.md

AWS Gen AI

CodeWhisperer

Amazon free AI code assistent, also provide 50 per month code security analysis

Setup CodeWhisperer

1. Install on VSCode the extension
2. Connect your local machine with AWS services by logging
3. Go back to VSCode and try example to check out the AWS CodeWhisperer introduction tutorial

Bedrock

With Amazon Bedrock you can train (fine tune) an AI giving a simple input like a PDF file and then create a chatbot to ask things about this PDF, also you can consume this created AI via API.

BILLED BY TOKEN USAGE

Creating a bedrock access policy

1. Go to IAM policies permissions
2. Click on the tab JSON
3. insert there:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "BedrockConsole",
            "Effect": "Allow",
            "Action": [  
               "bedrock:ListFoundationModels",
               "bedrock:GetFoundationModel",
               "bedrock:InvokeModel", 
               "bedrock:InvokeModelWithResponseStream", 
               "bedrock:CreateModelCustomizationJob", 
               "bedrock:GetModelCustomizationJob", 
               "bedrock:GetFoundationModelAvailability",
               "bedrock:ListModelCustomizationJobs", 
               "bedrock:StopModelCustomizationJob", 
               "bedrock:GetCustomModel", 
               "bedrock:ListCustomModels", 
               "bedrock:DeleteCustomModel",
               "bedrock:CreateProvisionedModelThroughput", 
               "bedrock:UpdateProvisionedModelThroughput", 
               "bedrock:GetProvisionedModelThroughput", 
               "bedrock:DeleteProvisionedModelThroughput", 
               "bedrock:ListProvisionedModelThroughputs", 
               "bedrock:ListTagsForResource", 
               "bedrock:UntagResource", 
               "bedrock:TagResource", 
               "bedrock:CreateAgent", 
               "bedrock:UpdateAgent", 
               "bedrock:GetAgent", 
               "bedrock:ListAgents", 
               "bedrock:CreateAgentActionGroup", 
               "bedrock:UpdateAgentActionGroup", 
               "bedrock:GetAgentActionGroup", 
               "bedrock:ListAgentActionGroups", 
               "bedrock:PrepareAgent", 
               "bedrock:GetAgentVersion",
               "bedrock:ListAgentVersions", 
               "bedrock:CreateAgentAlias", 
               "bedrock:UpdateAgentAlias",               
               "bedrock:GetAgentAlias",
               "bedrock:ListAgentAliases",
               "bedrock:InvokeAgent",  
               "bedrock:PutFoundationModelEntitlement",
               "bedrock:GetModelInvocationLoggingConfiguration",
               "bedrock:PutModelInvocationLoggingConfiguration",
               "bedrock:CreateFoundationModelAgreement",
               "bedrock:DeleteFoundationModelAgreement",
               "bedrock:ListFoundationModelAgreementOffers",
               "bedrock:GetUseCaseForModelAccess",
               "bedrock:PutUseCaseForModelAccess"
            ],
            "Resource": "*"
        }
    ]   
}         

4. Click Next going to Review and create
5. Set the Policy name, e.g. bedrock_policy
6. Click create policy

Creating a user with policies

1. Go to users tab
2. Click create user
3. On Specify user details give it a Name e.g. bedrock_user
4. Click Next going to Set permissions
5. Click on tab Attach policies directly
6. Search for your policy e.g. bedrock_policy and select it
7. Next > Review tab > Create

Requesting AI pre-defined models

1. Go to bedrock models access page
2. Click on Manage models access
3. Select the ones that you want
4. Click on Save changes

Acessing Bedrock programmatically

1. Install AWS CLI
2. Go to your created bedrock_user and generate an access key (users tab)
3. On terminal execute aws configure and enter the key and secret from your access key generated above
4. Code to test bedrock access:

import boto3

print(boto3.__version__)
boto_session = boto3.Session()
credentials = boto_session.get_credentials()

bedrock_models = boto3.client('bedrock')
print(bedrock_models.list_foundation_models())

Bedrock workshops

Cases to practice bedrock solutions!

Some term for bedrock answers configuration

• token: some words represented, this is used to estimate costs based on prompt text, response text
• temperature: lower = assertive responses || higher = creative responses
• length: maximum tokens on output

Sagemaker

Allows to personalize deeper Gen AI models!

BILLED BY INSTANCE HOUR

⚠️ Don't let instances (computers) turned on!! ⚠️

Creating a sagemaker

1. Create a sagemaker domain
2. Create a user for the domain created above
3. Select a jumpstart model

Refs

• Sagemaker Workshop
• Sagemaker Studio Jumpstart - Docs
• Sagemaker 101 Workshop

Tips when reducing costs

1. Begin the instance with the cheapest one
2. Configure autoscallling (when needed the infrastructure automatically changes to a better one)

Deploying AWS solution

Refs

• AWS - Documentação sobre precificação
• AWS - Documentação sobre o Inferentia 2
• AWS - Documentação sobre o Trainium

Scaling (bonus)

Which AI should you use?

1. Turn a big file into smaller ones
2. Transform them in vectors
3. Use it to fine tune