guia_desenvolviment.md

🧱 Guia de Desenvolvimento – Infraestrutura como Código (AWS CLI + JSON)

✅ Pré-requisitos

Antes de começar:

• AWS CLI instalado e configurado (aws configure)
• Permissões de administrador (IAM Full Access, S3, ECR, ECS, etc.)
• Linguagem padrão: bash
• Estrutura de pastas do projeto:

infra/
├─ policies/
├─ buckets/
├─ stepfunctions/
├─ ecs/
├─ eventbridge/
├─ parameters/
├─ secrets/
├─ logs/
├─ buildspecs/

---

1️⃣ CodeCommit + CodeBuild + ECR

1.1 Criar repositório no CodeCommit

aws codecommit create-repository \
  --repository-name sefaz-retrain-ml \
  --repository-description "Repositório de código de retreinamento SEFAZ"

1.2 Criar repositório no ECR

aws ecr create-repository \
  --repository-name sefaz-retrain-ml-container \
  --image-scanning-configuration scanOnPush=true \
  --encryption-configuration encryptionType=AES256

1.3 Criar Role para CodeBuild

Arquivo: policies/codebuild-role-trust-policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "codebuild.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

aws iam create-role \
  --role-name sefaz-ml-codebuild-role \
  --assume-role-policy-document file://policies/codebuild-role-trust-policy.json

1.4 Criar política de permissões para CodeBuild

Arquivo: policies/codebuild-policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "codecommit:Get*",
        "codecommit:GitPull",
        "ecr:*",
        "logs:*",
        "s3:*"
      ],
      "Resource": "*"
    }
  ]
}

aws iam put-role-policy \
  --role-name sefaz-ml-codebuild-role \
  --policy-name CodeBuildPermissions \
  --policy-document file://policies/codebuild-policy.json

1.5 Criar projeto CodeBuild

aws codebuild create-project \
  --name sefaz-ml-build-training-container \
  --source type=CODECOMMIT,location=https://git-codecommit.us-east-1.amazonaws.com/v1/repos/sefaz-retrain-ml \
  --artifacts type=NO_ARTIFACTS \
  --environment type=LINUX_CONTAINER,image=aws/codebuild/standard:5.0,computeType=BUILD_GENERAL1_SMALL,privilegedMode=true \
  --service-role arn:aws:iam::<ACCOUNT_ID>:role/sefaz-ml-codebuild-role \
  --description "Build da imagem de retreinamento ML para SEFAZ"

---

2️⃣ Armazenamento – Amazon S3

2.1 Criar buckets S3

BUCKET="sefaz-ml-data-prod"

aws s3api create-bucket \
  --bucket $BUCKET \
  --region us-east-1

# Habilitar versionamento
aws s3api put-bucket-versioning \
  --bucket $BUCKET \
  --versioning-configuration Status=Enabled

# Ativar criptografia
aws s3api put-bucket-encryption \
  --bucket $BUCKET \
  --server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}'

# Bloquear acesso público
aws s3api put-public-access-block \
  --bucket $BUCKET \
  --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

---

3️⃣ IAM Roles para Execução (ECS, Step Functions)

3.1 ECS Task Role

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "ssm:GetParameter", "secretsmanager:GetSecretValue", "logs:*"],
      "Resource": "*"
    }
  ]
}

aws iam create-role \
  --role-name sefaz-ml-ecs-task-role-prod \
  --assume-role-policy-document file://policies/ecs-task-role-trust.json

aws iam put-role-policy \
  --role-name sefaz-ml-ecs-task-role-prod \
  --policy-name ECSExecutionPolicy \
  --policy-document file://policies/ecs-execution-policy.json

3.2 Step Functions Role

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["ecs:RunTask", "ecs:DescribeTasks", "iam:PassRole", "logs:*"],
      "Resource": "*"
    }
  ]
}

aws iam create-role \
  --role-name sefaz-ml-stepfunctions-role \
  --assume-role-policy-document file://policies/stepfunctions-trust-policy.json

aws iam put-role-policy \
  --role-name sefaz-ml-stepfunctions-role \
  --policy-name StepFunctionsPolicy \
  --policy-document file://policies/stepfunctions-permissions.json

---

4️⃣ Orquestração – AWS Step Functions

4.1 Criar definição da State Machine

Arquivo: stepfunctions/retrain-model-definition.json

{
  "Comment": "Pipeline de retreinamento",
  "StartAt": "PreProcessData",
  "States": {
    "PreProcessData": {
      "Type": "Task",
      "Resource": "arn:aws:states:::ecs:runTask.sync",
      "Parameters": {
        "LaunchType": "FARGATE",
        "Cluster": "sefaz-ml-cluster-prod",
        "TaskDefinition": "sefaz-ml-task-preprocessing",
        "NetworkConfiguration": {
          "AwsvpcConfiguration": {
            "Subnets": ["subnet-xxxxx"],
            "AssignPublicIp": "DISABLED"
          }
        }
      },
      "Next": "TrainModel"
    },
    "TrainModel": {
      "Type": "Task",
      "Resource": "arn:aws:states:::ecs:runTask.sync",
      "Parameters": {
        "LaunchType": "FARGATE",
        "Cluster": "sefaz-ml-cluster-prod",
        "TaskDefinition": "sefaz-ml-task-training"
      },
      "Next": "Validate"
    },
    "Validate": {
      "Type": "Task",
      "Resource": "arn:aws:states:::ecs:runTask.sync",
      "Parameters": {
        "TaskDefinition": "sefaz-ml-task-validation"
      },
      "Next": "PublishArtifacts"
    },
    "PublishArtifacts": {
      "Type": "Task",
      "Resource": "arn:aws:states:::ecs:runTask.sync",
      "Parameters": {
        "TaskDefinition": "sefaz-ml-task-publish"
      },
      "Next": "Notify"
    },
    "Notify": {
      "Type": "Task",
      "Resource": "arn:aws:states:::sns:publish",
      "Parameters": {
        "TopicArn": "arn:aws:sns:us-east-1:<ACCOUNT_ID>:sefaz-ml-alerts-prod",
        "Message": "Retreinamento concluído com sucesso."
      },
      "End": true
    }
  }
}

aws stepfunctions create-state-machine \
  --name sefaz-ml-retrain-pipeline \
  --definition file://stepfunctions/retrain-model-definition.json \
  --role-arn arn:aws:iam::<ACCOUNT_ID>:role/sefaz-ml-stepfunctions-role

---

5️⃣ EventBridge – Agendamentos

5.1 Criar regra de agendamento semanal

aws events put-rule \
  --name sefaz-ml-retrain-schedule \
  --schedule-expression "cron(0 3 ? * MON *)"

5.2 Associar Step Function à regra

aws events put-targets \
  --rule sefaz-ml-retrain-schedule \
  --targets "Id"="1","Arn"="arn:aws:states:us-east-1:<ACCOUNT_ID>:stateMachine:sefaz-ml-retrain-pipeline"

---

6️⃣ Parameter Store e Secrets Manager

6.1 Criar parâmetros

aws ssm put-parameter \
  --name "/sefaz-ml/prod/training/batch_size" \
  --value "64" \
  --type "String"

aws ssm put-parameter \
  --name "/sefaz-ml/prod/training/epochs" \
  --value "30" \
  --type "String"

6.2 Criar segredos

aws secretsmanager create-secret \
  --name sefaz-ml/prod/db-credentials \
  --secret-string '{"username":"admin","password":"senhaforte"}'

---

7️⃣ Observabilidade – CloudWatch Logs e Métricas

7.1 Criar alarmes de falha

aws cloudwatch put-metric-alarm \
  --alarm-name "sefaz-ml-low-accuracy" \
  --metric-name ModelAccuracy \
  --namespace SEFAZ/MLModel \
  --statistic Average \
  --period 300 \
  --threshold 0.90 \
  --comparison-operator LessThanThreshold \
  --evaluation-periods 1 \
  --alarm-actions arn:aws:sns:us-east-1:<ACCOUNT_ID>:sefaz-ml-alerts-prod

---

8️⃣ Conclusão: Checklist Final

Item	Status ✅
Repositório CodeCommit criado	✅
Pipeline CodeBuild definido	✅
Bucket S3 com versionamento	✅
Role IAM para ECS e Step Func.	✅
Step Function com 5 etapas	✅
Evento do EventBridge criado	✅
Parâmetros no SSM definidos	✅
Secrets seguros configurados	✅
Logs e alarmes CloudWatch ativos	✅