Skip to content

Instantly share code, notes, and snippets.

@shaunhess

shaunhess/aws_cli_qrg.md

Last active February 11, 2020 13:01
Show Gist options
  • Save shaunhess/1d3200954057a6c20f0b7112e5044126 to your computer and use it in GitHub Desktop.
Save shaunhess/1d3200954057a6c20f0b7112e5044126 to your computer and use it in GitHub Desktop.
Download ZIP
AWS Command Line Interface (CLI) Quick Reference Guide
Raw
aws_cli_qrg.md

AWS Command Line Interface Quick Reference

The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts.

AWS CLI Credential Order

Order Prod Use Option Description
1 ❌ Command Line Options aws [command] --profile [profile name] - Profile data uses long term credentials stored locally. This is not recommended for production or instances with public access.
2 ❌ Environment Variables You can store values in environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN. Recommended for temp use in non-production environments.
3 ❌ AWS CLI Credentials File aws configure - Command creates a credentials file stored in ~/.aws/credentials on Linux, MacOSm or Unix, or at C:\Users\USERNAME\.aws\credentials on Windows. THis approach uses long term credential stored locally and is not recommended for production or instances with public access.
4 ✔️ Container Credentials IAM roles associated with AWS Elastic Container Service (ECS) Task Definitions. Temp credentials are available to the Task's containers. This is recommended for ECS environments.
5 ✔️ Instance Profile Credentials IAM roles assiciated with Elastic Compute Cloud (EC2) instances via Instance Profiles. Temp credentials are available to the instance. This is recommended for EC2 environments.

Prereqs

Install AWS CLI https://aws.amazon.com/cli/

Named Profiles

Creds and options stored in ~/.aws/credentials (Linux & Mac) or %USERPROFILE%\.aws\credentials (Windows)

aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-east-1
Default output format [None]: json

To configure multiple accounts, modify:

~/.aws/credentials (Linux & Mac) or %USERPROFILE%\.aws\credentials (Windows)

[default]
aws_access_key_id=AKIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

[user1]
aws_access_key_id=AKIAI44QH8DHBEXAMPLE
aws_secret_access_key=je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY

~/.aws/config (Linux & Mac) or %USERPROFILE%\.aws\config (Windows)

[default]
region=us-west-2
output=json

[profile user1]
region=us-east-1
output=text

To use different profiles:

Linux or Mac

export AWS_PROFILE=user1

Windows

setx AWS_PROFILE user1

On cmd line

aws s3 ls --profile user1

S3 - Simple Storage Service

Make a bucket

aws s3 mb s3://bucketname

List buckets

aws s3 ls

List content of bucket

aws s3 ls s3://bucketname

Copy a file to bucket

aws s3 cp filename.txt s3://bucketname

Sync folder content to S3

aws s3 sync .\local_folder s3://bucketname/folder_name

S3 - Bucket Policy to default everything public (ex. website content)

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "PublicReadGetObject",
      "Effect": "Allow",
      "Principal": "*",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:s3:::BUCKET_NAME/*"
      ]
    }
  ]
}

Using EC2 Roles and Instance Profiles

In this example, we have a VPC containing a bastion host and webserver. We want to add a development role for S3 access from our webserver. DO NOT STORE AWS ACCESS KEYS on instances exposed to the public (use roles).

Open a terminal on the bastion host withing the AWS VPC and create a file:

vi trust_policy_ec2.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {"Service": "ec2.amazonaws.com"},
      "Action": "sts:AssumeRole"
    }
  ]
}

Create the DEV IAM Role

aws iam create-role --role-name DEV_ROLE --assume-role-policy-document file://trust_policy_ec2.json

Create the IAM Policy for the S3 Dev Bucket Read Acess

vi dev_s3_read_access.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
          "Sid": "AllowUserToSeeBucketListInTheConsole",
          "Action": ["s3:ListAllMyBuckets", "s3:GetBucketLocation"],
          "Effect": "Allow",
          "Resource": ["arn:aws:s3:::*"]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:Get*",
                "s3:List*"
            ],
            "Resource": [
                "arn:aws:s3:::<DEV_S3_BUCKET_NAME>/*",
                "arn:aws:s3:::<DEV_S3_BUCKET_NAME>"
            ]
        }
    ]
}

aws iam create-policy --policy-name DevS3ReadAccess --policy-document file://dev_s3_read_access.json

Attach permissions to the DEV_ROLE

aws iam attach-role-policy --role-name DEV_ROLE --policy-arn "<POLICY_ARN_FROM_LAST_STEP>"

Verify settings

Verify the managed policy was attached:

aws iam list-attached-role-policies --role-name DEV_ROLE

Get the policy details, including the current version (be sure to replace <POLICY_ARN_FROM_LAST_STEP> with the policy ARN from earlier):

aws iam get-policy --policy-arn "<POLICY_ARN_FROM_LAST_STEP>"

Get the permissions associated with the current policy version (be sure to replace the <POLICY_ARN> and <DEFAULT_VERSION_ID> with the output of the get-policy command):

aws iam get-policy-version --policy-arn "<POLICY_ARN>" --version-id "<DEFAULT_VERSION_ID>"

Create the DEV instance Profile and add the DEV_ROLE

Create the instance profile:

aws iam create-instance-profile --instance-profile-name DEV_PROFILE

Add role to the new instance profile:

aws iam add-role-to-instance-profile --instance-profile-name DEV_PROFILE --role-name DEV_ROLE

Verify the configuration:

aws iam get-instance-profile --instance-profile-name DEV_PROFILE

Attach the DEV_PROFILE to an instance

Attach the DEV_PROFILE to an EC2 instance (be sure to replace the <LAB_WEB_SERVER_INSTANCE_ID> with the instance ID of the web server in your lab):

aws ec2 associate-iam-instance-profile --instance-id <LAB_WEB_SERVER_INSTANCE_ID> --iam-instance-profile Name="DEV_PROFILE"

Verify the configuration (be sure to replace the <LAB_WEB_SERVER_INSTANCE_ID> with the instance ID of the web server in your lab):

aws ec2 describe-instances --instance-ids <LAB_WEB_SERVER_INSTANCE_ID>

Test DEV_ROLE Permissions

Log into the web server instance

Determine the identity currently used:

aws sts get-caller-identity

Verify access to the <DEV_S3_BUCKET_NAME> in your lab (be sure to replace the <DEV_S3_BUCKET_NAME> with the value provided in your lab):
aws s3 ls
aws s3 ls s3://<DEV_S3_BUCKET_NAME>

Verify access is denied to the <SECRET_S3_BUCKET_NAME> (be sure to replace the <SECRET_S3_BUCKET_NAME> with the value provided in your lab):

aws s3 ls s3://<SECRET_S3_BUCKET_NAME>

Running Docker on EC2 (recommend ECS or EKS instead)

sudo amazon-linux-extras install docker
sudo service docker start
sudo usermod -a -G docker ec2-user

sudo yum install git
git clone https://github.com/<YOUR_GIT_REPO>

cd YOUR_GIT_REPO
docker build -t <name> .
docker images --filter reference=<name>
docker run -t -i -p 80:80 <name>

Upload docker image to dockerhub registry

docker login --username YOUR_USER
docker images
docker tag IMAGEID YOUR_USER/<name>
docker push YOUR_USER/<name>

Stress test EC2 Instance (Useful to test auto scaling)

sudo amazon-linux-extras install epel -y
sudo yum install stress -y
stress --cpu 2 --timeout 30000

SQS - Simple Queue Service

Get Queue Attributes

aws sqs get-queue-attributes --queue-url https://URL --attribute-names All

Send Message

aws sqs send-message --queue-url https://URL --message-body "INSERTMESSAGE"

Receive Message

aws sqs receive-message --wait-time-seconds 10 --max-number-of-messages 10 --queue-url https://URL aws sqs --region us-east-1 receive-message --wait-time-seconds 10 --max-number-of-messages 10 --queue-url https://URL

Delete Message

aws sqs delete-message --queue-url https://URL --receipt-handle "INSERTHANDLE"

KMS - Key Management Service

Create a Custom Managed Key (CMK)

aws kms create-key --description "DEVALEX KMS DEMO CMK"

Create an alias -- Point app to use alias since each key is region specific (duplicate alias in each region)

aws kms create-alias --target-key-id XXX --alias-name "alias/devalexkmsdemo" --region us-east-1

Encrypt/Decrypt text (up to 4KB)

Windows
echo "this is a secret message" topsecret.txt
aws kms encrypt --key-id KEYID --plaintext file://topsecret.txt --output text --query CiphertextBlob 
aws kms encrypt --key-id KEYID --plaintext file://topsecret.txt --output text --query CiphertextBlob > topsecret.base64.encrypted
certutil -decode topsecret.base64.encrypted topsecret.encrypted

aws kms decrypt --ciphertext-blob fileb://topsecret.encrypted --output text --query Plaintext > topsecret.decrypted.base64
certutil topsecret.decrypted.base64 topsecret.decrypted
Linux/MacOS
echo "this is a secret message" topsecret.txt
aws kms encrypt --key-id KEYID --plaintext file://topsecret.txt --output text --query CiphertextBlob 
aws kms encrypt --key-id KEYID --plaintext file://topsecret.txt --output text --query CiphertextBlob | base64 --decode > topsecret.encrypted
aws kms decrypt --ciphertext-blob fileb://topsecret.encrypted --output text --query Plaintext | base64 --decode

Generate a Data Encryption Key (DEK) - can be used to encrypt data > 4KB

aws kms generate-data-key --key-id KEYID --key-spec AES_256 --region us-east-1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment