How to set up for and use Packer's Amazon Import post-processor

gistfile1.txt

Packer has the ability to import Virtualbox OVA artifacts into AWS
using Amazons Import/Export feature. There are a number of prerequisites,
the setup of which are detailed below.

1. Create an S3 bucket to hold uploaded temporary build artifacts

$ aws s3 mb s3://my-organisation.com-eu-west-1-vmimport-bucket --region eu-west-1
make_bucket: s3://my-organisation.com-eu-west-1-vmimport-bucket/


2. Set up the VM Import Service Role. This is a special role within the
   AWS account that users can assume to perform VM imports and exports. It
   is used internally by the AWS import service rather than being attached
   to a specific user or group. Note that the user performing the import
   will still need to be granted required permissions as shown below

2.1 Create the role policy document trust-policy.json with the following
    contents

{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"",
      "Effect":"Allow",
      "Principal":{
        "Service":"vmie.amazonaws.com"
      },
      "Action":"sts:AssumeRole",
      "Condition":{
        "StringEquals":{
          "sts:ExternalId":"vmimport"
        }
      }
    }
  ]
}

2.2 Now create the role using the AWS cli

$ aws iam create-role --role-name vmimport --assume-role-policy-document file:///path/to/trust-policy.json
{
  "Role": {
    "CreateDate": "2016-07-25T10:42:49.317Z",
    "Path": "/",
    "Arn": "arn:aws:iam::993562413670:role/vmimport",
    "RoleId": "AROAJVAKXW5OAPT2FI4LI",
    "RoleName": "vmimport",
    "AssumeRolePolicyDocument": {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "",
          "Action": "sts:AssumeRole",
          "Condition": {
            "StringEquals": {
              "sts:ExternalId": "vmimport"
            }
          },
          "Effect": "Allow",
          "Principal": {
            "Service": "vmie.amazonaws.com"
          }
        }
      ]
    }
  }
}

2.3 Create a policy file with the required permissions on the bucket
    created in step 1 e.g. with contents as below, and name it
    role-policy.json.

{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Effect":"Allow",
      "Action":[
        "s3:ListBucket",
        "s3:GetBucketLocation"
      ],
      "Resource":[
        "arn:aws:s3:::my-organisation.com-eu-west-1-vmimport-bucket"
      ]
    },
    {
      "Effect":"Allow",
      "Action":[
        "s3:GetObject"
      ],
      "Resource":[
        "arn:aws:s3:::my-organisation.com-eu-west-1-vmimport-bucket/*"
      ]
    },
    {
      "Effect":"Allow",
      "Action":[
        "ec2:ModifySnapshotAttribute",
        "ec2:CopySnapshot",
        "ec2:RegisterImage",
        "ec2:Describe*"
      ],
      "Resource":"*"
    }
  ]
}



2.4 Run the following command to associate the role policy with the
    vmimport role

$ aws iam put-role-policy --role-name vmimport --policy-name vmimport --policy-document file:///path/to/role-policy.json


3. Create a group and set up the required permissions on EC2 and S3 object
   needed to upload and convert VM images to AMI's

3.1 Create a VMImport group

$ aws iam create-group --group-name VMImport

3.2 Now create a policy document with the required permissions and name it
    vmimportexport-group-policy.json as per the contents below

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListAllMyBuckets"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:CreateBucket",
        "s3:DeleteBucket",
        "s3:DeleteObject",
        "s3:GetBucketLocation",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:PutObject"
      ],
      "Resource": ["arn:aws:s3:::my-organisation.com-eu-west-1-vmimport-bucket","arn:aws:s3:::my-organisation.com-vmimport-bucket/*"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:CancelConversionTask",
        "ec2:CancelExportTask",
        "ec2:CreateImage",
        "ec2:CreateInstanceExportTask",
        "ec2:CreateTags",
        "ec2:DeleteTags",
        "ec2:DescribeConversionTasks",
        "ec2:DescribeExportTasks",
        "ec2:DescribeInstanceAttribute",
        "ec2:DescribeInstanceStatus",
        "ec2:DescribeInstances",
        "ec2:DescribeTags",
        "ec2:ImportInstance",
        "ec2:ImportVolume",
        "ec2:StartInstances",
        "ec2:StopInstances",
        "ec2:TerminateInstances",
        "ec2:ImportImage",
        "ec2:ImportSnapshot",
        "ec2:DescribeImportImageTasks",
        "ec2:DescribeImportSnapshotTasks",
        "ec2:CancelImportTask",
        "ec2:DescribeImageAtrribute",
        "ec2:DescribeImages"
      ],
      "Resource": "*"
    }
  ]
}

3.3 Create the policy

$ aws iam create-policy --policy-name "AWSVMImportExportOnly" --policy-document file:///path/to/vmimportexport-group-policy.json
{
    "Policy": {
        "IsAttachable": true,
        "Path": "/",
        "PolicyId": "ANPAI6BFYVZC6XHH6QZ5G",
        "DefaultVersionId": "v1",
        "Arn": "arn:aws:iam::993562413670:policy/AWSVMImportExportOnly",
        "AttachmentCount": 0,
        "UpdateDate": "2016-07-25T11:58:20.737Z",
        "PolicyName": "AWSVMImportExportOnly",
        "CreateDate": "2016-07-25T11:58:20.737Z"
    }
}

3.4 Attach the policy to the group

$ aws iam attach-group-policy --group-name VMImport --policy-arn "arn:aws:iam::993562413670:policy/AWSVMImportExportOnly"


4. Create a user and make that user a member of the group created above

4.1 Create the user

$ aws iam create-user --user-name packer

4.2 Add the user to the VMImport group

$ aws iam add-user-to-group --user-name packer --group-name VMImport

4.3 Create an access key for the user. This will create the AWS 'Access
    Key ID' and 'Secret Access Key'

$ aws iam create-access-key --user-name packer


5. The AWS Access Key ID and Secret Access Key must be made available to
   packer. The accepted way to do this is to export the ID and Access Key
   as environment variables that packer can then be configured to read and
   use in the 'amazon-import' post-processor.
   Example template below:

{
  "variables": {
    ...
    "aws_access_key": "{{env `PACKER_AWS_ACCESS_KEY`}}",
    "aws_secret_key": "{{env `PACKER_AWS_SECRET_KEY`}}",
    "template": "centos"
    ...
  },

  ...

  "post-processors": [
    {
      "type": "amazon-import",
      "only": ["amazon-builder"],
      "access_key": "{{user `aws_access_key`}}",
      "secret_key": "{{user `aws_secret_key`}}",
      "region": "eu-west-1",
      "s3_bucket_name": "my-organisation.com-eu-west-1-vmimport-bucket",
      "s3_key_name": "{{user `template`}}-import-{{isotime \"2006-01-02-150405\"}}.ova",
      "skip_clean": "false",
      "tags": {
        "Description": "Packer amazon-import: {{user `template`}} {{isotime \"2006-01-02 15:04:05\"}}",
        "Name": "{{user `template`}}"
      }
    }
  ]
}


Enable Amazon Import Feature for Additional Regions
---------------------------------------------------

1. Create a bucket for the desired region

$ aws s3 mb s3://my-organisation.com-eu-west-2-vmimport-bucket --region eu-west-2


2. Delete the existing role policy document associated with the vmimport
   role

$ aws iam delete-role-policy --role-name vmimport --policy-name vmimport


3. Add the bucket created in step 1 to the 'role-policy.json' file

        ...
        "s3:GetBucketLocation"
      ],
      "Resource":[
        "arn:aws:s3:::my-organisation.com-eu-west-1-vmimport-bucket",
        "arn:aws:s3:::my-organisation.com-eu-west-2-vmimport-bucket"
      ]
    },
    {
      "Effect":"Allow",
      "Action":[
        "s3:GetObject"
      ],
      "Resource":[
        "arn:aws:s3:::my-organisation.com-eu-west-1-vmimport-bucket/*",
        "arn:aws:s3:::my-organisation.com-eu-west-2-vmimport-bucket/*"
        ...


4. Associate the updated role policy with the vmimport role

$aws iam put-role-policy --role-name vmimport --policy-name vmimport --policy-document file:///path/to/role-policy.json


5. Detach the existing AWSVMImportExportOnly group policy from the
   VMImport group

$ aws iam detach-group-policy --group-name VMImport --policy-arn "arn:aws:iam::993562413670:policy/AWSVMImportExportOnly"


6. Delete the policy

$ aws iam delete-policy --policy-arn "arn:aws:iam::993562413670:policy/AWSVMImportExportOnly"


7. Update the group policy file with the updated bucket list

        ...
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::my-organisation.com-eu-west-1-vmimport-bucket",
        "arn:aws:s3:::my-organisation.com-eu-west-1-vmimport-bucket/*",
        "arn:aws:s3:::my-organisation.com-eu-west-2-vmimport-bucket",
        "arn:aws:s3:::my-organisation.com-eu-west-2-vmimport-bucket/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:CancelConversionTask",
        ...


8. Recreate the policy


$ aws iam create-policy --policy-name "AWSVMImportExportOnly" --policy-document file:///path/to/vmimportexport-group-policy.json
{
    "Policy": {
        "Arn": "arn:aws:iam::993562413670:policy/AWSVMImportExportOnly",
        "PolicyId": "ANPAJLETXFWZ6MQUYVFL6",
        "AttachmentCount": 0,
        "PolicyName": "AWSVMImportExportOnly",
        "Path": "/",
        "CreateDate": "2016-12-15T15:30:37.805Z",
        "DefaultVersionId": "v1",
        "UpdateDate": "2016-12-15T15:30:37.805Z",
        "IsAttachable": true
    }
}


9. Reattach the policy to the VMImport group

$ aws iam attach-group-policy --group-name VMImport --policy-arn "arn:aws:iam::993562413670:policy/AWSVMImportExportOnly"