thieryl/01_aws_rbd_cli.md

Last active March 31, 2022 10:49

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/thieryl/01f0d0551aabba656edd30c15323120f.js"></script>
Save thieryl/01f0d0551aabba656edd30c15323120f to your computer and use it in GitHub Desktop.

Download ZIP

[AWS] Amazon Web Services #aws

Raw

01_aws_rbd_cli.md

AWS CLI RBD

White list office ip

aws allow-my-ip "Proxy-Cerebro" tcp 22 rbd_sys

Raw

02_aws_boto.md

boto3 quick hands-on

This documentation aims at being a quick-straight-to-the-point-hands-on AWS resources manipulation with boto3.

First of all, you'll need to install boto3. Installing it along with awscli is probably a good idea as

awscli is boto-based
awscli usage is really close to boto's
boto3 will use the same configuration files

A convenient method consists in installing them in a python virtualenv:

$ virtualenv awscli
$ . awscli/bin/activate
(awscli)$ pip install awscli boto3

From now on I will assume you already have an AWS account setup, best practices suggest that you must NOT use your master account but instead create an account which might have adequate privileges.

$ aws configure
AWS Access Key ID [None]: your_key_id
AWS Secret Access Key [None]: your_access_key
Default region name [None]: eu-west-1
Default output format [None]: json

In order switch between regions more easily, I suggest you make aliases on the awscli configuration files:

$ cat ~/.aws/credentials 
[default]
aws_access_key_id = your_key_id
aws_secret_access_key = your_access_key

[ireland]
aws_access_key_id = your_key_id
aws_secret_access_key = your_access_key

$ cat ~/.aws/config
[default]
output = json
region = eu-west-1

[profile ireland]
output=json
region=eu-west-1

You should then be able to summon awscli by region:

$ aws ec2 describe-instances --profile ireland
{
    [some json output]
}

We are now able to dig around with boto3 itself. In order to learn its usage, we'll use the fantastic ipython. First import boto3:

$ ipython3
In [1]: import boto3

Then initiate a boto session:

In [2]: s = boto3.Session(profile_name='ireland')

From there, initiate an object corresponding to the resource you want to manipulate, for instance ec2:

In [3]: ec2 = s.resource('ec2')

And witness the methods at your disposal:

In [4]: ec2.[press tab]
ec2.DhcpOptions                    ec2.create_snapshot
ec2.Image                          ec2.create_subnet
ec2.Instance                       ec2.create_tags
ec2.InternetGateway                ec2.create_volume
ec2.KeyPair                        ec2.create_vpc
ec2.NetworkAcl                     ec2.create_vpc_peering_connection
ec2.NetworkInterface               ec2.dhcp_options_sets
ec2.PlacementGroup                 ec2.disassociate_route_table
ec2.RouteTable                     ec2.images
ec2.RouteTableAssociation          ec2.import_key_pair
ec2.SecurityGroup                  ec2.instances
ec2.Snapshot                       ec2.internet_gateways
ec2.Subnet                         ec2.key_pairs
ec2.Tag                            ec2.meta
ec2.Volume                         ec2.network_acls
ec2.Vpc                            ec2.network_interfaces
ec2.VpcPeeringConnection           ec2.placement_groups
ec2.create_dhcp_options            ec2.register_image
ec2.create_instances               ec2.route_tables
ec2.create_internet_gateway        ec2.security_groups
ec2.create_key_pair                ec2.snapshots
ec2.create_network_acl             ec2.subnets
ec2.create_network_interface       ec2.volumes
ec2.create_placement_group         ec2.vpc_peering_connections
ec2.create_route_table             ec2.vpcs
ec2.create_security_group

While those methods permit to execute operations, they also give access to objects themselves, for example:

In [5]: for i in ec2.instances.all(): print(i)
ec2.Instance(id='i-ea39b240')
ec2.Instance(id='i-54c4fafe')
ec2.Instance(id='i-0d0533a7')
ec2.Instance(id='i-44c5fbee')
ec2.Instance(id='i-9405333e')
ec2.Instance(id='i-b105331b')
ec2.Instance(id='i-04fc68ae')

Let's pick one of those:

In [6]: i = ec2.Instance(id='i-ea39b240')

And discover its methods:

In [7]: i.[press tab]
i.ami_launch_index          i.private_ip_address
i.architecture              i.product_codes
i.attach_classic_link_vpc   i.public_dns_name
i.attach_volume             i.public_ip_address
i.block_device_mappings     i.ramdisk_id
i.client_token              i.reboot
i.console_output            i.reload
i.create_image              i.report_status
i.create_tags               i.reset_attribute
i.describe_attribute        i.reset_kernel
i.detach_classic_link_vpc   i.reset_ramdisk
i.detach_volume             i.reset_source_dest_check
i.ebs_optimized             i.root_device_name
i.hypervisor                i.root_device_type
i.iam_instance_profile      i.security_groups
i.id                        i.source_dest_check
i.image                     i.spot_instance_request_id
i.image_id                  i.sriov_net_support
i.instance_id               i.start
i.instance_lifecycle        i.state
i.instance_type             i.state_reason
i.kernel_id                 i.state_transition_reason
i.key_name                  i.stop
i.key_pair                  i.subnet
i.launch_time               i.subnet_id
i.load                      i.tags
i.meta                      i.terminate
i.modify_attribute          i.unmonitor
i.monitor                   i.virtualization_type
i.monitoring                i.volumes
i.network_interfaces        i.vpc
i.password_data             i.vpc_id
i.placement                 i.wait_until_exists
i.placement_group           i.wait_until_running
i.platform                  i.wait_until_stopped
i.private_dns_name          i.wait_until_terminated

For example:

In [8]: i.hypervisor
Out[8]: 'xen'

Here's an example of the filter method that many objects use to match only certain criterias:

In [9]: filter = {'Name': 'name', 'Values' : ['debian*amd64*ebs']}
In [10]: for i in ec2.images.filter(Filters = [filter]): print(i)
ec2.Image(id='ami-61e56916')
ec2.Image(id='ami-879e4ff0')
ec2.Image(id='ami-8bf29ffc')
ec2.Image(id='ami-971a65e0')
ec2.Image(id='ami-99f39eee')
ec2.Image(id='ami-c935cbbe')
ec2.Image(id='ami-e31a6594')
ec2.Image(id='ami-e7e66a90')

Accessing services through resource is called the high level method, and not everything is yet reachable through this consistent interface. The other, low-level method is client:

In [11]: ec2c = s.client('ec2')
In [12]: ec2c.[press tab]
Display all 188 possibilities? (y or n)
ec2c.accept_vpc_peering_connection
ec2c.allocate_address
ec2c.assign_private_ip_addresses
ec2c.associate_address
ec2c.associate_dhcp_options
ec2c.associate_route_table
ec2c.attach_classic_link_vpc
ec2c.attach_internet_gateway
ec2c.attach_network_interface
ec2c.attach_volume
ec2c.attach_vpn_gateway
ec2c.authorize_security_group_egress
ec2c.authorize_security_group_ingress
ec2c.bundle_instance
ec2c.can_paginate
[...]

As a usage example, no high-level resource is available to list a region Availability Zones, but a client can achieve this:

In [13]: ec2c.describe_availability_zones()
Out[13]:
{'AvailabilityZones': [{'Messages': [],
   'RegionName': 'eu-west-1',
   'State': 'available',
   'ZoneName': 'eu-west-1a'},
  {'Messages': [],
   'RegionName': 'eu-west-1',
   'State': 'available',
   'ZoneName': 'eu-west-1b'},
  {'Messages': [],
   'RegionName': 'eu-west-1',
   'State': 'available',
   'ZoneName': 'eu-west-1c'}],
 'ResponseMetadata': {'HTTPStatusCode': 200,
  'RequestId': '46e474a5-7e77-4716-a9a1-010990d066ee'}}

Every single resource and client methods are documented in boto3's documentation.

Last but not least, I wrote a convenient little module in order to ease boto3 usage. It is available in my GitHub repository.

Its usage is pretty straightforward:

In [14]: from session import Aws
In [15]: ec2 = Aws('ireland', 'ec2')

The instanciated ec2 objects holds both resource and client methods along with helper functions and variables:

In [16]: ec2.[press tab]
ec2.change_nsrecord      ec2.getamis              ec2.profile
ec2.client               ec2.getinst              ec2.region
ec2.create_tag           ec2.gettagval            ec2.resource
ec2.dmesg                ec2.lsinstances          ec2.session
ec2.get_id_from_nametag  ec2.lsinstnames          ec2.tags2dict
ec2.getall               ec2.mktags               
ec2.getami               ec2.mkuserdata

Every helper is nicely documented:

In [17]: ec2.change_nsrecord??
[...]
    def change_nsrecord(self, action, dnsrecord):
        '''Create, delete or modify a DNS record

        :param str action: One of ``CREATE``, ``DELETE`` or ``UPSERT``
        :param dict dnsrecord: A dict describing the DNS record to change
[...]

And uses boto3 functions behind the scenes:

In [18]: ec2.dmesg('i-ea39b240').split('\n')[0]
Out[18]: u'ian.net/debian/ wheezy/main debconf-utils all 1.5.49 [55.8 kB]\r'

Let's finish this quick hands on with a very basic EC2 instance creation and termination using boto3:

In [19]: rc = ec2.resource.create_instances(
    ImageId = ec2.getami('NetBSD*64*6.1.5*'),
    MinCount = 1,
    MaxCount = 1,
    KeyName = 'mysshpemkey',
    InstanceType = 'm3.medium',
    PrivateIpAddress = '10.10.0.1',
    SubnetId = ec2.get_id_from_nametag('subnets', 'examplesubnet')
)
In [20]: print(rc[0].id)
i-b1774f1b
In [21]: rc[0].terminate()
Out[21]: 
{'ResponseMetadata': {'HTTPStatusCode': 200,
  'RequestId': '45738798-812c-4ea4-9c15-edb7ddcd9950'},
 u'TerminatingInstances': [{u'CurrentState': {u'Code': 32,
    u'Name': 'shutting-down'},
   u'InstanceId': 'i-b1774f1b',
   u'PreviousState': {u'Code': 16, u'Name': 'running'}}]}

In this example, we retrieve the AMI id through my module's getami() function, and the SubnetId using my helper function get_id_from_nametag() which supposes that you gave a tag, examplesubnet here, to the subnet you intend to spawn this instance in.

The create_instances function returns an array of instances objects that are immediately ready for usage, and as a matter of fact, in this example, we show the instance id and terminate it.

Hope that quick hands on has been informative, do not hesitate to comment and share it!

Emile iMil Heitor -- [email protected]

Raw

AWS_Certifications.md

A curated list of AWS resources to prepare for the AWS Certifications

A curated list of awesome AWS resources you need to prepare for the all 5 AWS Certifications. This gist will include: open source repos, blogs & blogposts, ebooks, PDF, whitepapers, video courses, free lecture, slides, sample test and many other resources.

OP: @leonardofed founder @ plainflow.

Index:

Passing the AWS solutions architect - Associate exam (Published ☑)

Exam Overview
Prerequisites
General Learning Material
Blueprints exam
Direct experience from AWS Certified members
The exam

Passing the AWS solutions architect - Professional exam (Published ☑)

Exam Overview
Prerequisites
General Learning Material
Blueprints exam
Direct experience from AWS Certified members
The exam

Passing the AWS solutions architect - Associate exam > Exam Overview

You will find you make less errors when you don’t feel rushed on time.

The AWS Certified Solutions Architect – Associate exam is intended for individuals with experience designing distributed applications and systems on the AWS platform.

Exam concepts you should understand for this exam include:

Designing and deploying scalable, highly available, and fault tolerant systems on AWS
Lift and shift of an existing on-premises application to AWS
Ingress and egress of data to and from AWS
Selecting the appropriate AWS service based on data, compute, database, or security requirements
Identifying appropriate use of AWS architectural best practices
Estimating AWS costs and identifying cost control mechanisms

Passing the AWS solutions architect - Associate exam > Prerequisites & Requirements

Candidate Overview description provided by the AWS documentation

Eligible candidates for this exam have:

- One or more years of hands-on experience designing available, cost efficient, fault tolerant, and scalable distributed systems on AWS
- In-depth knowledge of at least one high-level programming language
- Ability to identify and define requirements for an AWS-based application
- Experience with deploying hybrid systems with on-premises and AWS components
- Capability to provide best practices for building secure and reliable applications on the AWS platform

AWS Knowledge required for the Exam:

Hands-on experience using compute, networking, storage, and database AWS services
Professional experience architecting large-scale distributed systems
Understanding of elasticity and scalability concepts
Understanding of the AWS global infrastructure
Understanding of network technologies as they relate to AWS
A good understanding of all security features and tools that AWS provides and how they relate to traditional services
A strong understanding of client interfaces to the AWS platform
Hands-on experience with AWS deployment and management services

Key items you should know before you take the exam:

How to configure and troubleshoot a VPC inside and out, including basic IP subnetting. VPC is arguably one of the more complex components of AWS and you cannot pass this exam without a thorough understanding of it.
The difference in use cases between Simple Workflow (SWF), Simple Queue Services (SQS), and Simple Notification Services (SNS).
How an Elastic Load Balancer (ELB) interacts with auto-scaling groups in a high-availability deployment.
How to properly secure a S3 bucket in different usage scenarios
When it would be appropriate to use either EBS-backed or ephemeral instances.
A basic understanding of CloudFormation.
How to properly use various EBS volume configurations and snapshots to optimize I/O performance and data durability.

General IT Knowledge preferred for the Exam:

Excellent understanding of typical multi-tier architectures: web servers, caching, application servers, load balancers, and storage
Understanding of Relational Database Management System (RDBMS) and NoSQL
Knowledge of message queuing and Enterprise Service Bus (ESB)
Familiarity with loose coupling and stateless systems
Understanding of different consistency models in distributed systems
Knowledge of Content Delivery Networks (CDN)
Hands-on experience with core LAN/WAN network technologies
Experience with route tables, access control lists, firewalls, NAT, HTTP, DNS, IP and OSI Network
Knowledge of RESTful Web Services, XML, JSON
Familiarity with the software development lifecycle
Work experience with information and application security concepts, mechanisms, and tools
Awareness of end-user computing and collaborative technologies

Passing the AWS solutions architect - Associate exam > General Learning Material

Passing the AWS solutions architect - Associate exam > Blueprints exam

In this AWS whitepaper you'll find a sample exam. Here's a preview:

Amazon Glacier is designed for: (Choose 2 answers)

B. infrequently accessed data.

C. data archives.

Your web application front end consists of multiple EC2 instances behind an Elastic Load Balancer. You configured ELB to perform health checks on these EC2 instances. If an instance fails to pass health checks, which statement will be true?

C. The ELB stops sending traffic to the instance that failed its health check.

You are building a system to distribute confidential training videos to employees. Using CloudFront, what method could be used to serve content that is stored in S3, but not publically accessible from S3 directly?

A. Create an Origin Access Identity (OAI) for CloudFront and grant access to the objects in your S3 bucket to that OAI.

In this amazing post Markos Rendell gave a deep explanation to every single AWS question.

Passing the AWS solutions architect - Associate exam > Direct experience from AWS Certified members

Here are some general observations by Miha Kralj in this great post.

There were several questions related to DR solutions with specified RPO/RTO times. Modern cloud-born solutions use completely different BCP approach, but hey, someone in AWS really likes traditional disaster recovery scenarios and is making sure that you love them too. I know it is 2016, but you need to learn the old skool BCP techniques for this exam.
Questions about the AWS Storage Gateway appear at least 3 times. Yeah. Storage Gateway. The stuff that cloud-native architects never saw in action - nor do we want to. You have to learn the difference between Cached Volumes, Stored Volumes and understand how VTL works.
Lots and lots and lots of questions on deployment management. CloudFormation. Elastic Beanstalk. OpsWorks. Learn these three technologies well - not well for an architect, but well for a 2nd-tier escalation operations engineer. One of the examiners really really really liked cloud deployment automation. And now you will like it too. Who cares if you use SaltStack, Terraform or Ansible - learn CF, Beanstalk and OpsWorks!
Networking questions were everywhere, like 30% of the test or even more: VPN/DirectConnect/VPC peering. For me, DDOS protection, WAF, Cloudfront, and SSL/TLS stuff is networking too, although AWS treats them as security issues. Anyway, the examiners *love* networking. Learn networking. I mean, learn it like this is a Cisco exam, not a cloud architecture exam. Federated access, SAML, IAM roles and all possible AuthZ/AuthN scenarios - learn them all. Learn how IAM policies work. How cross-account trust works. And specifically how they don't work. Think like troubleshooting support personnel and what they need to know about identity flows; that's what you need to know for this exam.
Whenever you see the need for high-performing scalable solution, the answer is always DynamoDB. Even if you think that architecturally there might be a better choice (Cassandra, or CouchDB anyone?), the correct answer will be DynamoDB. People that wrote the test were clearly in love with DynamoDB, Elasticache and Kinesis. Just pick the answer that includes all three of them and you'll be right.
If a scenario is asking for something cheap (cost-effective), the answer must include spot instances, SQS for throttling and perhaps S3 RRS or Glacier.
There were at least two questions where I was simply forced to propose the AWS Data Pipeline. Yeah, the obscure and rarely-seen Data Pipeline service, in the age when Lambda solves the same problem way more efficiently. No, Lambda was not an option at all and it didn't appear anywhere in the test.

-- In this other write-up on LinkedIn Wong Chun Yin explained how to get all 5 AWS Certifications in Asia. Below, a couple of great hints for the SA - Associate.

First of all, associate certifications are not hard, and if you have a chance to take the AWS training, then you just need to concentrate on reading the training slides is more than enough! Remember to read the details explanation under the slides. Good understanding of VPC and IAM is important for all associate exams.

Dan-Claudiu Dragos shared his experience here on how he prepared for the AWS Solutions Architect Certifications in 7 days and succesfully passed it.

I'd like to share my experience of getting AWS CSA(A) certified with Cloud Academy:

The background:

I have registered my personal AWS account late 2014 and still do not do much with it. Without a professional motivator this is actually a dead end, more like buying a book and never reading it.
Mid-2015 I started doing DevOps work for a customer of my employer. They have a 1000+ node AWS environment that was fully configured with multiple VPCs, VPN access, IAM groups and the like. That become my playground and was the actual game changer, the single big detail that made the difference, certification-wise.

The process:

Late April 2016 I have found the r/sysadmin (reddit) message with one month promotion by Cloud Academy. At that point I did not know anything about the AWS certifications but the seed got planted. I found the message a bit late, though, when the seats were already filled up, so did not register at that time.
During the first week or so afterwards I was a bit confused, did not know what path to take. My first intention was to go to the Sysops cert but then I read on the Cloud Academy page that there is a big overlap with the "simpler" Architecture certification.
By looking around I have found some course recordings from 2 years ago (don't ask) and listened to them for a total of 14 to 16 hours (not sure about this detail). They helped me get in the right mood to start doing tests, quizes, practice stuff...
Mid-May, I register myself with Cloud Academy and get the 7 days trial. Well, I did the best out of that - my public profile says I have completed 1600+ quizes and got 35,000+ karma during that time. I have taken every quiz from the then AWS CSA(A) learning path multiple times until I got my score above 90%. The EC2/EBS quizes were quite easy, actually, with my experience; the S3 and IAM ones were average and Cloud Academy helped me fill in many blanks in that area. The database ones (DynamoDB and RDS) were the hardest and I had to open a lab to see how things were done and what concepts were important.
In the last 3 days of the trial I have taken the 150+ questions exam at the end of the learning path and got 75% on the first try. I have taken it 2 or 3 more times, but as I started to remember questions I no longer considered it that useful to figure out what I still don't know.
By that time I have also started to read white papers from Amazon on topics that were lightly touched by Cloud Academy, e.g. EBS RAID configurations and Route 53 special record types, health checks and failovers.
I also got 4 apps from the Google Play Store, I found "AWS Architect - Associate" and "Cloud Pros- AWS Certified Arch" best. At that point I was already above the 90% passing threshold, though, and could not find many questions online I could not provide the expected correct answer to.
I have also taken the practice exam from Amazon (a $20 cost). Please note that the questions do not change so taking it once and taking photos of the screen really helps on figuring out the failed questions. Nevertheless, I got 90% and scheduled myself a slot for "the real thing".
May 27th - I went to the testing center and passed the AWS CSA(A) exam with 83%; I assume this is an average passing score. Amazon doesn't tell what is the failing threshold, but tells you how well you did in 4 areas (I had 80-85-90% in all of them).

On the exam itself:

I got one question from the sample questions and one from the practice exam; they were on the simpler side.
33% are "easy", in the sense that fall in the "is water wet? true/false" type - relative to the AWS concepts, though.
33% are "average", more like "what feels wetter, water or oil?"
33% are hard or even crazy, covering all sorts of service details or requiring you to provide answers in the line of Amazon recommendations regarding certain service usage.
About half of them are multiple answer, with no partial points given.
Oh, don't look for dumps, Amazon has hundreds of possible questions out of which a subset is being given for each exam, there simply is no way to pass such exam with brain dumps, so forget it.

That's it. I'm the number 16.891, not sure if this is small or big, or even if it matters.

A redditor on r/aws gave awesome tips about the exam day

Passing the AWS solutions architect - Associate exam > The Exam

Exam Registration fee is USD 150
You have 80 minutes to complete a 40 quizzes exam. Most of the questions are up to 3 lines long in the multiple choice format. You should consider no more than 1.5/2 minutes per question if you want to read each question carefully and answer to all of them correctly.
It's possible to set a question for review and skip, you can get back to what you marked in this way at the end.

Now you're ready to go. Here's where you book your exam!

Passing the AWS solutions architect - Professional Exam > Exam Overview

This is a curated list of hands-on material to help you passing this AWS Certification! This advanced list of selected points are especially for students who already have a working knowledge of AWS and who have passed the Solutions Architect - Associate Certification for AWS exam (prerequisite for sitting the Solutions Architect - Professional Certification for AWS exam). This should be helpful to build and develop your skills as an AWS professional.

Exam Overview

Multiple choice and multiple answer questions
170 minutes to complete the exam. It's all multiple choice on a PC
Exam available in English and Japanese
Practice Exam Registration fee is USD 40
The Exam blueprint specified that there would be 100+ questions given in a 180 minute period and did not specify a pass grade.
The exam will test your knowledge with 80 questions
In terms of question complexity, it requires a good understanding of all available AWS services
AWS Certification passing scores are set by using statistical analysis and are subject to change. AWS does not publish exam passing scores because exam questions and passing scores are subject to change without notice
Exam Registration fee is USD 300
Recommend taking Advanced Architecting on AWS
Sample questions for the exam are available here.

What should I bring to an AWS Certification exam?

Candidates must show two forms of personal identification (ID). Primary form must be a valid, government-issued ID containing both a photo and signature. The secondary form of ID needs to be valid and contain a signature.
Acceptable Forms of Primary ID (name, photograph, signature, valid/current):

Government-issued Driver’s license
U.S. Department of State Driver’s License
National/State/Country Identification Card
Passport
Passport cards
Military ID
Alien Registration Card (Green Card, Permanent Resident Visa)

Note: Irish natives may use a Public Services Card as a primary form of identification, in Ireland only.

Acceptable forms of Secondary ID (name, signature, valid/current):
U.S. Social Security Card
Debit/(ATM) Card
Credit Card
School ID (without a signature for minors is acceptable) any form of ID on the primary list

Note: In Japan, the blue colored (not pink) Health Insurance Card is an acceptable form of secondary identification.
However, the paper form of the Health Insurance is not acceptable.
You can NOT bring food, laptops, backpacks, notepads, or other personal equipment to the test area. For all exams, you can request a whiteboard and marker (some centers may hand out paper and pencil), which must be returned before you leave. During check in you’ll be asked to turn out your pockets (on jackets, pants, etc.) to verify they’re empty and free of prohibited items. Eyewear will also be inspected to ensure that it’s not technology-enabled.

Passing the AWS solutions architect - Professional exam > Prerequisites & Requirements

To be eligible for this exam, you must already be certified at the AWS Certified Solutions Architect – Associate Level. You should have multiple years of hands-on experience designing and deploying cloud architecture on AWS, along with the ability to evaluate cloud application requirements and make architectural recommendations for implementation, deployment, and provisioning applications on AWS. Additionally, you should have the experience and the capability to provide best practices guidance on the architectural design across multiple applications, projects, or the enterprise.

Note that in the event that you fail to pass an AWS certification exam, you may retake the exam subject to the following conditions:

a. You must wait 14 days from the day you fail to take the exam again
b. You can take an exam up to three times in one year from the date of your first attempt

This is valid for any AWS Certifiation Exam.

To pass the AWS Certified Solutions Architect - Professional exam, you have to master advanced and technical skills, not to mention the experience in designing distributed applications and systems using AWS. Check the short list below to understand you need to master in order to pass the exam.

Exam concepts you should understand for this exam include:

Designing and deploying dynamically scalable, highly available, fault tolerant, and reliable applications on AWS
Selecting appropriate AWS services to design and deploy an application based on given requirements
Migrating complex, multi-tier applications on AWS
Designing and deploying enterprise-wide scalable operations on AWS
Implementing cost control strategies

Candidate Overview

This exam tests your knowledge of advanced AWS use cases. Eligible candidates for this exam have:

Achieved AWS Certified Solutions Architect - Associate
2+ years hands-on experience designing and deploying cloud architecture on AWS
Abilities to evaluate cloud application requirements and make architectural recommendations for implementation, deployment, and provisioning applications on AWS.
Capabilities to provide best practices guidance on the architectural design across multiple applications, projects, or the enterprise.

Key Points to pass the Exam:

Demonstrate ability to architect the appropriate level of availability based on stakeholder requirements

Stakeholder requirements is key phrase here – look at what the requirements are first before deciding the best way to architect the solution
What is availability? Basically up time. Does the customer need 99.99% up time or less? Which products may need to be used to meet this requirement?
Look at products which are single AZ, multi AZ and multi region. It may be the case that a couple of instances in a single AZ will suffice if cost is a factor
CloudWatch can be used to perform EC2 or auto scaling actions when status checks fail or metrics are exceeded (alarms, etc)

Demonstrate ability to implement DR for systems based on RPO and RTO

What is DR? It is the recovery of systems, services and applications after an unplanned period of downtime.
What is RPO? Recovery Point Objective. At which point in time do we need to get back to when DR processes are invoked? 3. 3. This would come from a customer requirement – when systems are recovered, data is consistent from 30 minutes prior to the outage, or 1 hour, or 4 hours etc. What is acceptable to the stakeholder?
What is RTO? Recovery Time Objective. How quickly must systems and services be recovered after invoking DR processes? It may be that all critical systems must be back online within a maximum of four hours.
RTO and RPO are often paired together to provide an SLA to end users as to when services will be fully restored and how much data may be lost. For example, an RTO of 2 hours and an RPO of 15 minutes would mean all systems would be recovered in two hours or less and consistent to within 15 minutes of the failure.
How can low RTO be achieved? This can be done by using elastic scaling, for example or using monitoring scripts to power up new instances using the AWS API. You may also use multi AZ services such as EBS and RDS to provide additional resilience
How can low RPO be achieved? This can be done by using application aware and consistent backup tools, usually native ones such as VSS aware ones from Microsoft or RMAN for Oracle, for example. Databases and real time systems may need to be acquiesced to obtain a crash consistent backup. Standard snapshot tools may not provide this. RMAN can backup to S3 or use point in time snapshots using RDS. RMAN is supported on EC2. Use data dump to move large databases.
AWS has multi AZ, multi region and services like S3 which has 11 nines of durability with cross region replication
Glacier – long term archive storage. Cheap but not appropriate for fast recovery (several hours retrieval SLA)
Storage Gateway is a software appliance that sits on premises that can operate in three modes – gateway cached (hot data kept locally but most data stored in S3), gateway stored (all data kept locally but also replicated to S3) and VTL-Tape Library (virtual disk tapes stored in S3, virtual tape shelf stored in Glacier)
You should use gateway cached when the requirement is for low cost primary storage with hot data stored locally
Gateway stored keeps all data locally but takes asynchronous snapshots to S3
Gateway cached volumes can store 32TB of data, 32 volumes are supported (32 x 32, 1PB)
Gateway stored volumes are 16TB in size, 12 volumes are supported (16 x 12, 192TB)
Virtual tape library supports 1500 virtual tapes in S3 (150 TB total)
Virtual tape shelf is unlimited tapes (uses Glacier)
Storage Gateway can be on premises or EC2. Can also schedule snapshots, supports Direct Connect and also bandwidth throttling.
Storage Gateway supports ESXi or Hyper-V, 7.5GB RAM, 75GB storage, 4 or 8 vCPU for installation. To use the Marketplace appliance, you must choose xlarge instance or bigger and m3, i2, c3, c4, r3, d2, or m4 instance types
Gateway cached requires a separate volume as a buffer upload area and caching area
Gateway stored requires enough space to hold your full data set and also an upload buffer VTL also requires an upload buffer and cache area
Ports required for Storage Gateway include 443 (HTTPS) to AWS, port 80 for initial activation only, port 3260 for iSCSI internally and port 53 for DNS (internal)
Gateway stored snapshots are stored in S3 and can be used to recover data quickly. EBS snapshots can also be used to create a volume to attach to new EC2 instances
Can also use gateway snapshots to create a new volume on the gateway itself
Snapshots can also be used to migrate cached volumes into stored volumes, stored volumes into cached volumes and also snapshot a volume to create a new EBS volume to attach to an instance
Use System Resource Check from the appliance menu to ensure the appliance has enough virtual resources to run (RAM, vCPU, etc.)
VTL virtual tape retrieval is instantaneous, whereas Tape Shelf (Glacier) can take up to 24 hours
VTL supports Backup Exec 2012-15, Veeam 7 and 8, NetBackup 7, System Center Data Protection 2012, Dell NetVault 10
Snapshots can either be scheduled or done ad hoc
Writes to S3 get throttled as the write buffer gets close to capacity – you can monitor this with CloudWatch
EBS – Elastic Block Store – block based storage replicated across hosts in a single AZ in a region
Direct Connect – connection directly into AWS’s data centre via a trusted third party. This can be backed up with standby Direct Connect links or even software VPN
Route53 also has 100% uptime SLA, Elastic Load Balancing and VPC can also provide a level of resilience if required
DynamoDB has three copies per region and also can perform multi-region replication
RDS also supports multi-AZ deployments and read only replicas of data. 5 read only replicas for MySQL, MariaDB and PostGres, 15 for Aurora
There are four DR models in the AWS white paper:

Backup and restore (cheap but slow RPO and RTO, use S3 for quick restores and AWS Import/Export for large datasets)
Pilot Light (minimal replication of the live environment, like the pilot light in a gas heater, it’s used to bring services up with the smallest footprint running in DR. AMIs ready but powered off, brought up manually or by autoscaling
Data must be replicated to DR from the primary site for failover)
Warm Standby (again a smaller replication of the live environment but with some services always running to facilitate a quicker failover. It can also be the full complement of servers but running on smaller instances than live. Horizontal scaling is preferred to add more instances to a load balancer)
Multi-site (active/active configuration where DNS sends traffic to both sites simultaneously. Auto scaling can also add instances for load where required. DNS weighting can be used to route traffic accordingly). DNS weighting is done as a percentage, so if two records have weightings of 10, then the overall is 20 and the percentage is 50% chance of either being used, this is round robin. Weights of 10 and 40 would mean a total of weight 50, with 1 in 5 chance of weight 10 DNS record being used

Import/Export can import data sets into S3, EBS or Glacier. You can only export from S3
Import/Export makes sense for large datasets that cannot be moved or copied into AWS over the internet in an efficient manner (time, cost, etc)
AWS will export data back to you encrypted with TrueCrypt
AWS will wipe devices after import if specified
If exporting from an S3 bucket with versioning enabled, only the most recent version is exported
Encryption for imports is optional, mandatory for exports
Some services have automated backup:

RDS
Redshift
Elasticache (Redis only)

EC2 does not have automated backup. You can use either EBS snapshots or create an AMI Image from a running or stopped instance. The latter option is especially useful if you have an instance storage on the host which is ephemeral and will get deleted when the instance is stopped (Bundle Instance). You can “copy” the host storage for the instance by creating an AMI, which can then be copied to another region
To restore a file on a server for example, take regular snapshots of the EBS volume, create a volume from the snapshot, mount the volume to the instance, browse and recover the files as necessary
MySQL requires InnoDB for automated backups, if you delete an instance then all automated backups are deleted, manual DB snapshots stored in S3 are not deleted
All backups are stored in S3 When you do an RDS restore, you can change the engine type (SQL Standard to Enterprise, for example), assuming you have enough storage space.
Elasticache automated backups snapshot the whole cluster, so there will be performance degradation whilst this takes place. Backups are stored on S3.
Redshift backups are stored on S3 and have a 1 day retention period by default and only backs up delta changes to keep storage consumption to a minimum
EC2 snapshots are stored in S3 and are incremental and each snapshot still contains the base snapshot data. You are only charged for the incremental snapshot storage

Determine appropriate use of multi-Availability Zones vs. multi-Region architectures

Multi-AZ services examples are S3, RDS, DynamoDB. Using multi-AZ can mitigate against the loss of up to two AZs (data centres, assuming there are three. Some regions only have two). This can provide a good balance between cost, complexity and reliability
Multi-region services can mitigate failures in AZs or individual regions, but may cost more and introduce more infrastructure and complexity. Use ELB for multi-region failover and resilience, CloudFront
DynamoDB offers cross region replication, RDS offers the ability to snapshot from one region to another to have read only replicas. Code Pipeline has a built in template for replicating DynamoDB elsewhere for DR
Redshift can snapshot within the same region and also replicate to another region

Demonstrate ability to implement self-healing capabilities

HA available already for most popular databases:-
SQL Server Availability Groups, SQL Mirroring, log shipping. Read replicas in other AZs not supported
MySQL – Asynchronous mirroring
Oracle – Data Guard, RAC (RAC not supported on AWS but can run on EC2 by using VPN and Placement Groups as multicast is not supported)
RDS has multi-AZ automatic failover to protect against
Loss of availability in primary AZ
Loss of connectivity to primary DB
Storage or host failure of primary DB
Software patching (done by AWS, remember)
Rebooting of primary DB
Uses master and slave model
MySQL, Oracle and Postgres use physical layer replication to keep data consistent on the standby instance
SQL Server uses application layer mirroring but achieves the same result
Multi-AZ uses synchronous replication (consistent read/write), asynchronous (potential data loss) is only used for read replicas
DB backups are taken from the secondary to reduce I/O load on the primary
DB restores are taken from the secondary to avoid I/O suspension on the primary
AZ failover can be forced by rebooting your instance either via the console or via the RebootDBInstance API call
Multi-AZ databases are used for DR, not as a scaling solution. Scale can be achieved by using read replicas, this can be done via the AWS console or by using the CreateDBInstanceReadReplica API call
Amazon Aurora employs a highly durable, SSD-backed virtualized storage layer purpose-built for database workloads.
Amazon Aurora automatically replicates your volume six ways, across three Availability Zones. Amazon Aurora storage is fault-tolerant, transparently handling the loss of up to two copies of data without affecting database write availability and up to three copies without affecting read availability. Amazon Aurora storage is also self-healing. Data blocks and disks are continuously scanned for errors and replaced automatically.
Creating a read replica means a snapshot of your primary DB instance, this may result in a pause of about a minute in non multi-AZ deployments
Multi-AZ deployments will use a secondary for a snapshot
A new DNS endpoint address is given for the read only replica, you need to update the app
You can promote a read only replica to be a standalone, but this breaks replication
MySQL and Postgres can have up to 5 replicas
Read replicas in different regions for MySQL only
Replication is asynchronous only
Read replicas can be built off Multi-AZ databases
Read replicas are not multi-AZ
MySQL can have read replicas of read replicas, but this increases latency
DB Snapshots and automated backups cannot be taken of read replicas
Consider using DynamoDB instead of RDS if your database does not require:-
Transaction support
Atomicity
Consistency
Isolation
Durability
ACID (durability) compliance
Joins
SQL

Credits to Chris Beckett @ BlueClouds

Passing the AWS solutions architect - Professional exam > General Learning Material
To prepare at best for the exam you should start with an overview of the concepts and knowledge areas covered on the exam and walks you through the exam structure and question formats. Get an hands-on practice with advanced use cases, while practice exam questions test your understanding of key architectural concepts.

Solutions Architect— Professional Certification for AWS (2016)

A Guide to AWS Certification Exams

AWS Best Practices for DDoS Resiliency

Amazon Kinesis Streams FAQs

Amazon Kinesis Streams: Developer Guide

AWS Identity and Access Management User Guide

Amazon CloudFront - Dynamic Content Delivery

Amazon Redshift FAQs

Amazon AWS Certified Solutions Architect: What to Study, Tips and Resources

VPC Migration: NATs & Bandwidth Bottleneck

Amazon Simple Notification Service Developer Guide

Amazon Simple Storage Service Developer Guide

Amazon Virtual Private Cloud User Guide

13. Best Practices for Migrating from RDBMS to Amazon DynamoDB 14. AWS Certification Exams: What to expect at the Exam 15. Enabling Federation to AWS Using Windows Active Directory, ADFS, and SAML 2.0 16. AWS Certified Solutions Architect – Professional Level Sample Exam Questions 17. AWS Certified Solutions Architect Professional Exam Blueprint 18. Cloud Architectures with AWS Direct Connect (ARC304) | AWS re:Invent 2013 19. AWS re: Invent STG 204: Using AWS Storage Gateway 20. Storage TCO using AWS Storage Gateway, Amazon S3 and Amazon Glacier (STG202) | AWS re:Invent 2013 21. AWS re:Invent 2014 | (ARC206) Architecting Reactive Applications on AWS (Playlist) 22. AWS June Webinar Series - Deep dive: Hybrid Architectures 23. AWS January 2016 Webinar Series - Managing your Infrastructure as Code 24. Complete AWS IAM Reference 25. Free Templates for AWS CloudFormation 26. How to study for the AWS Solutions Architect Professional Certification (Webinar) 27. Interview with 5 AWS Certified Greg Cockburn (Podcast) 28. Interview with 5 AWS Certified Stephen Wilding (Podcast)

Passing the AWS solutions architect - Professional Exam > Blueprints Exam

In this PDF you can download the Sample Question provided by AWS We reviewed all the questions provided by AWS and you can find the correct answers marked in bold.

Which AWS based disaster recovery strategy will give you the best RTO?

A) Deploy the Oracle database and the JBoss app server on EC2. Restore the RMAN Oracle backups from Amazon S3. Generate an EBS volume of static content from the Storage Gateway and attach it to the JBoss EC2 server.
B) Deploy the Oracle database on RDS. Deploy the JBoss app server on EC2. Restore the RMAN Oracle backups from Amazon Glacier. Generate an EBS volume of static content from the Storage Gateway and attach it to the JBoss EC2 server.
C) Deploy the Oracle database and the JBoss app server on EC2. Restore the RMAN Oracle backups from Amazon S3. Restore the static content by attaching an AWS Storage Gateway running on Amazon EC2 as an iSCSI volume to the JBoss EC2 server.
D) Deploy the Oracle database and the JBoss app server on EC2. Restore the RMAN Oracle backups from Amazon S3. Restore the static content from an AWS Storage Gateway-VTL running on Amazon EC2

An ERP application is deployed in multiple Availability Zones in a single region. In the event of failure, the RTO must be less than 3 hours, and the RPO is 15 minutes. The customer realizes that data corruption occurred roughly 1.5 hours ago. Which DR strategy can be used to achieve this RTO and RPO in the event of this kind of failure?

A) Take 15-minute DB backups stored in Amazon Glacier, with transaction logs stored in Amazon S3 every 5 minutes.
B) Use synchronous database master-slave replication between two Availability Zones.
C) Take hourly DB backups to Amazon S3, with transaction logs stored in S3 every 5 minutes.
D) Take hourly DB backups to an Amazon EC2 instance store volume, with transaction logs stored in Amazon S3 every 5 minutes.

The Marketing Director in your company asked you to create a mobile app that lets users post sightings of good deeds known as random acts of kindness in 80-character summaries. You decided to write the application in JavaScript so that it would run on the broadest range of phones, browsers, and tablets. Your application should provide access to Amazon DynamoDB to store the good deed summaries. Initial testing of a prototype shows that there aren’t large spikes in usage. Which option provides the most costeffective and scalable architecture for this application?

A) Provide the JavaScript client with temporary credentials from the Security Token Service using a Token Vending Machine (TVM) on an EC2 instance to provide signed credentials mapped to an Amazon Identity and Access Management (IAM) user allowing DynamoDB puts and S3 gets. You serve your mobile application out of an S3 bucket enabled as a web site. Your client updates DynamoDB.
B) Register the application with a Web Identity Provider like Amazon, Google, or Facebook, create an IAM role for that provider, and set up permissions for the IAM role to allow S3 gets and DynamoDB puts. You serve your mobile application out of an S3 bucket enabled as a web site. Your client updates DynamoDB.
C) Provide the JavaScript client with temporary credentials from the Security Token Service using a Token Vending Machine (TVM) to provide signed credentials mapped to an IAM user allowing DynamoDB puts. You serve your mobile application out of Apache EC2 instances that are load-balanced and autoscaled. Your EC2 instances are configured with an IAM role that allows DynamoDB puts. Your server updates DynamoDB.
D) Register the JavaScript application with a Web Identity Provider like Amazon, Google, or Facebook, create an IAM role for that provider, and set up permissions for the IAM role to allow DynamoDB puts. You serve your mobile application out of Apache EC2 instances that are load-balanced and autoscaled. Your EC2 instances are configured with an IAM role that allows DynamoDB puts. Your server updates DynamoDB.

You are building a website that will retrieve and display highly sensitive information to users. The amount of traffic the site will receive is known and not expected to fluctuate. The site will leverage SSL to protect the communication between the clients and the web servers. Due to the nature of the site you are very concerned about the security of your SSL private key and want to ensure that the key cannot be accidentally or intentionally moved outside your environment. Additionally, while the data the site will display is stored on an encrypted EBS volume, you are also concerned that the web servers’ logs might contain some sensitive information; therefore, the logs must be stored so that they can only be decrypted by employees of your company. Which of these architectures meets all of the requirements?

A) Use Elastic Load Balancing to distribute traffic to a set of web servers. To protect the SSL private key, upload the key to the load balancer and configure the load balancer to offload the SSL traffic. Write your web server logs to an ephemeral volume that has been encrypted using a randomly generated AES key.
B) Use Elastic Load Balancing to distribute traffic to a set of web servers. Use TCP load balancing on the load balancer and configure your web servers to retrieve the private key from a private Amazon S3 bucket on boot. Write your web server logs to a private Amazon S3 bucket using Amazon S3 server-side encryption.
C) Use Elastic Load Balancing to distribute traffic to a set of web servers, configure the load balancer to perform TCP load balancing, use an AWS CloudHSM to perform the SSL transactions, and write your web server logs to a private Amazon S3 bucket using Amazon S3 server-side encryption.
D) Use Elastic Load Balancing to distribute traffic to a set of web servers. Configure the load balancer to perform TCP load balancing, use an AWS CloudHSM to perform the SSL transactions, and write your web server logs to an ephemeral volume that has been encrypted using a randomly generated AES key.

You are designing network connectivity for your fat client application. The application is designed for business travelers who must be able to connect to it from their hotel rooms, cafes, public Wi-Fi hotspots, and elsewhere on the Internet. You do not want to publish the application on the Internet. Which network design meets the above requirements while minimizing deployment and operational costs?

A) Implement AWS Direct Connect, and create a private interface to your VPC. Create a public subnet and place your application servers in it.
B) Implement Elastic Load Balancing with an SSL listener that terminates the back-end connection to the application.
C) Configure an IPsec VPN connection, and provide the users with the configuration details. Create a public subnet in your VPC, and place your application servers in it.
D) Configure an SSL VPN solution in a public subnet of your VPC, then install and configure SSL VPN client software on all user computers. Create a private subnet in your VPC and place your application servers in it.

Your company hosts an on-premises legacy engineering application with 900GB of data shared via a central file server. The engineering data consists of thousands of individual files ranging in size from megabytes to multiple gigabytes. Engineers typically modify 5-10 percent of the files a day. Your CTO would like to migrate this application to AWS, but only if the application can be migrated over the weekend to minimize user downtime. You calculate that it will take a minimum of 48 hours to transfer 900GB of data using your company’s existing 45-Mbps Internet connection. After replicating the application’s environment in AWS, which option will allow you to move the application’s data to AWS without losing any data and within the given timeframe?

A) Copy the data to Amazon S3 using multiple threads and multi-part upload for large files over the weekend, and work in parallel with your developers to reconfigure the replicated application environment to leverage Amazon S3 to serve the engineering files.
B) Sync the application data to Amazon S3 starting a week before the migration, on Friday morning perform a final sync, and copy the entire data set to your AWS file server after the sync completes.
C) Copy the application data to a 1-TB USB drive on Friday and immediately send overnight, with Saturday delivery, the USB drive to AWS Import/Export to be imported as an EBS volume, mount the resulting EBS volume to your AWS file server on Sunday.
D) Leverage the AWS Storage Gateway to create a Gateway-Stored volume. On Friday copy the application data to the Storage Gateway volume. After the data has been copied, perform a snapshot of the volume and restore the volume as an EBS volume to be attached to your AWS file server on Sunday.

The Exam Day

Just a couple of suggestions to pass the AWS Solutions Architect Professional level certification:

Wake up early every day and have a nice breakfast in the morning

No coffee, try to eat a banana and a cup of orange juice every morning

Practice with our specific resources for the AWS Certifications

Drink water every day

Get the most out of this GitHub repository

Rewatch the AWS Solution Architect Professional level certification detailed study-guide with tips and tricks on how to pass the certification

AWS Youtube channel: watch as many of the technical seminars and re:Invent presentations as you can. Then re-watch. Make notes.

I would say that your chance of passing the course if you have not have had any practical experience to be seriously compromised. You have no excuse, get an account, login and play.

Do some sport every day to reset your mind out of work and try to sleep 7h every night

Repeat

Once you are feeling confident enough you are ready to take a practice exam here

Smile at least once every day

Have fun

If you've found this gist useful you can follow me @leonardofed for more info about AWS Certifications.

Raw

aws_cli_cheatsheet.md

AWS CLI Cheatsheet

http://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html https://www.youtube.com/watch?v=_wiGpBQGCjU

Setup

Overview

Virtualbox
Ubuntu 14.04 LTS VM, 64-bit http://releases.ubuntu.com/14.04/ubuntu-14.04.4-desktop-amd64.iso
create new machine, settings
- System / Processor
  - Enable PAE/NX
- System / Acceleration
  - Paravirtualization Interface: Default
  - Enable VT-x/AMD-V
  - Enable Nested Paging
- Display / Screen
  - Video Memory: 128MB
  - Acceleration: Enable 3D Acceleration
boot
install

install Virtualbox Guest Additions, passwordless sudo

echo $USER
sudo echo "$USER ALL=(ALL) NOPASSWD:ALL" | sudo tee -a /etc/sudoers
sudo su
apt-get update
apt-get install -y build-essential dkms linux-headers-$(uname -r)
cd /media/aws-admin/
sh ./VBoxLinuxAdditions.run
shutdown now

install AWS CLI

sudo apt-get install -y python-dev python-pip
sudo pip install awscli
aws --version
aws configure

Bash one-liners

cat <file> # output a file
tee # split output into a file
cut -f 2 # print the 2nd column, per line
sed -n '5{p;q}' # print the 5th line in a file
sed 1d # print all lines, except the first
tail -n +2 # print all lines, starting on the 2nd
head -n 5 # print the first 5 lines
tail -n 5 # print the last 5 lines

expand # convert tabs to 4 spaces
unexpand -a # convert 4 spaces to tabs
wc # word count
tr ' ' \\t # translate / convert characters to other characters

sort # sort data
uniq # show only unique entries
paste # combine rows of text, by line
join # combine rows of text, by initial column value

Cloudtrail - Logging and Auditing

http://docs.aws.amazon.com/cli/latest/reference/cloudtrail/ 5 Trails total, with support for resource level permissions

# list all trails
aws cloudtrail describe-trails

# list all S3 buckets
aws s3 ls

# create a new trail
aws cloudtrail create-subscription \
    --name awslog \
    --s3-new-bucket awslog2016

# list the names of all trails
aws cloudtrail describe-trails --output text | cut -f 8

# get the status of a trail
aws cloudtrail get-trail-status \
    --name awslog

# delete a trail
aws cloudtrail delete-trail \
    --name awslog

# delete the S3 bucket of a trail
aws s3 rb s3://awslog2016 --force

# add tags to a trail, up to 10 tags
aws cloudtrail add-tags \
    --resource-id awslog \
    --tags-list "Key=log-type,Value=all"

# list the tags of a trail
aws cloudtrail list-tags \
    --resource-id-list 

# remove a tag from a trail
aws cloudtrail remove-tags \
    --resource-id awslog \
    --tags-list "Key=log-type,Value=all"

IAM

Users

https://blogs.aws.amazon.com/security/post/Tx15CIT22V4J8RP/How-to-rotate-access-keys-for-IAM-users http://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam-limits.html Limits = 5000 users, 100 group, 250 roles, 2 access keys / user

http://docs.aws.amazon.com/cli/latest/reference/iam/index.html

# list all user's info
aws iam list-users

# list all user's usernames
aws iam list-users --output text | cut -f 6

# list current user's info
aws iam get-user

# list current user's access keys
aws iam list-access-keys

# crate new user
aws iam create-user \
    --user-name aws-admin2

# create multiple new users, from a file
allUsers=$(cat ./user-names.txt)
for userName in $allUsers; do
    aws iam create-user \
        --user-name $userName
done

# list all users
aws iam list-users --no-paginate

# get a specific user's info
aws iam get-user \
    --user-name aws-admin2

# delete one user
aws iam delete-user \
    --user-name aws-admin2

# delete all users
# allUsers=$(aws iam list-users --output text | cut -f 6);
allUsers=$(cat ./user-names.txt)
for userName in $allUsers; do
    aws iam delete-user \
        --user-name $userName
done

Password policy

http://docs.aws.amazon.com/cli/latest/reference/iam/

# list policy
# http://docs.aws.amazon.com/cli/latest/reference/iam/get-account-password-policy.html
aws iam get-account-password-policy

# set policy
# http://docs.aws.amazon.com/cli/latest/reference/iam/update-account-password-policy.html
aws iam update-account-password-policy \
	--minimum-password-length 12 \
	--require-symbols \
	--require-numbers \
	--require-uppercase-characters \
	--require-lowercase-characters \
	--allow-users-to-change-password

# delete policy
# http://docs.aws.amazon.com/cli/latest/reference/iam/delete-account-password-policy.html
aws iam delete-account-password-policy

Access Keys

http://docs.aws.amazon.com/cli/latest/reference/iam/

# list all access keys
aws iam list-access-keys

# list access keys of a specific user
aws iam list-access-keys \
    --user-name aws-admin2

# create a new access key
aws iam create-access-key \
    --user-name aws-admin2 \
    --output text | tee aws-admin2.txt

# list last access time of an access key
aws iam get-access-key-last-used \
    --access-key-id AKIAINA6AJZY4EXAMPLE

# deactivate an acccss key
aws iam update-access-key \
    --access-key-id AKIAI44QH8DHBEXAMPLE \
    --status Inactive \
    --user-name aws-admin2

# delete an access key
aws iam delete-access-key \
    --access-key-id AKIAI44QH8DHBEXAMPLE \
    --user-name aws-admin2

Groups, Policies, Managed Policies

http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html http://docs.aws.amazon.com/cli/latest/reference/iam/

# list all groups
aws iam list-groups

# create a group
aws iam create-group --group-name FullAdmins

# delete a group
aws iam delete-group \
    --group-name FullAdmins

# list all policies
aws iam list-policies

# get a specific policy
aws iam get-policy \
    --policy-arn <value>

# list all users, groups, and roles, for a given policy
aws iam list-entities-for-policy \
    --policy-arn <value>

# list policies, for a given group
aws iam list-attached-group-policies \
    --group-name FullAdmins

# add a policy to a group
aws iam attach-group-policy \
    --group-name FullAdmins \
    --policy-arn arn:aws:iam::aws:policy/AdministratorAccess

# add a user to a group
aws iam add-user-to-group \
    --group-name FullAdmins \
    --user-name aws-admin2

# list users, for a given group
aws iam get-group \
    --group-name FullAdmins

# list groups, for a given user
aws iam list-groups-for-user \
    --user-name aws-admin2

# remove a user from a group
aws iam remove-user-from-group \
    --group-name FullAdmins \
    --user-name aws-admin2

# remove a policy from a group
aws iam detach-group-policy \
    --group-name FullAdmins \
    --policy-arn arn:aws:iam::aws:policy/AdministratorAccess

# delete a group
aws iam delete-group \
    --group-name FullAdmins

S3

https://docs.aws.amazon.com/cli/latest/reference/s3api/index.html#cli-aws-s3api

# list existing S3 buckets
aws s3 ls

# create a bucket name, using the current date timestamp
bucket_name=test_$(date "+%Y-%m-%d_%H-%M-%S")
echo $bucket_name

# create a public facing bucket
aws s3api create-bucket --acl "public-read-write" --bucket $bucket_name

# verify bucket was created
aws s3 ls | grep $bucket_name

# check for public facing s3 buckets (should show the bucket name you created)

aws s3api list-buckets --query 'Buckets[*].[Name]' --output text | xargs -I {} bash -c 'if [[ $(aws s3api get-bucket-acl --bucket {} --query '"'"'Grants[?Grantee.URI==`http://acs.amazonaws.com/groups/global/AllUsers` && Permission==`READ`]'"'"' --output text) ]]; then echo {} ; fi'

# check for public facing s3 buckets, updated them to be private

aws s3api list-buckets --query 'Buckets[*].[Name]' --output text | xargs -I {} bash -c 'if [[ $(aws s3api get-bucket-acl --bucket {} --query '"'"'Grants[?Grantee.URI==`http://acs.amazonaws.com/groups/global/AllUsers` && Permission==`READ`]'"'"' --output text) ]]; then aws s3api put-bucket-acl --acl "private" --bucket {} ; fi'

# check for public facing s3 buckets (should be empty)

aws s3api list-buckets --query 'Buckets[*].[Name]' --output text | xargs -I {} bash -c 'if [[ $(aws s3api get-bucket-acl --bucket {} --query '"'"'Grants[?Grantee.URI==`http://acs.amazonaws.com/groups/global/AllUsers` && Permission==`READ`]'"'"' --output text) ]]; then echo {} ; fi'

EC2

keypairs

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html

# list all keypairs
# http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-key-pairs.html
aws ec2 describe-key-pairs

# create a keypair
# http://docs.aws.amazon.com/cli/latest/reference/ec2/create-key-pair.html
aws ec2 create-key-pair \
    --key-name <value> --output text

# create a new local private / public keypair, using RSA 4096-bit
ssh-keygen -t rsa -b 4096

# import an existing keypair
# http://docs.aws.amazon.com/cli/latest/reference/ec2/import-key-pair.html
aws ec2 import-key-pair \
    --key-name keyname_test \
    --public-key-material file:///home/apollo/id_rsa.pub

# delete a keypair
# http://docs.aws.amazon.com/cli/latest/reference/ec2/delete-key-pair.html
aws ec2 delete-key-pair \
    --key-name <value>

Security Groups

http://docs.aws.amazon.com/cli/latest/reference/ec2/index.html

# list all security groups
aws ec2 describe-security-groups

# create a security group
aws ec2 create-security-group \
    --vpc-id vpc-1a2b3c4d \
    --group-name web-access \
    --description "web access"

# list details about a securty group
aws ec2 describe-security-groups \
    --group-id sg-0000000

# open port 80, for everyone
aws ec2 authorize-security-group-ingress \
    --group-id sg-0000000 \
    --protocol tcp \
    --port 80 \
    --cidr 0.0.0.0/24

# get my public ip
my_ip=$(dig +short myip.opendns.com @resolver1.opendns.com);
echo $my_ip

# open port 22, just for my ip
aws ec2 authorize-security-group-ingress \
    --group-id sg-0000000 \
    --protocol tcp \
    --port 80 \
    --cidr $my_ip/24

# remove a firewall rule from a group
aws ec2 revoke-security-group-ingress \
    --group-id sg-0000000 \
    --protocol tcp \
    --port 80 \
    --cidr 0.0.0.0/24

# delete a security group
aws ec2 delete-security-group \
    --group-id sg-00000000

Images

https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-images.html

# list all private AMI's, ImageId and Name tags
aws ec2 describe-images --filter "Name=is-public,Values=false" \
    --query 'Images[].[ImageId, Name]' \
    --output text | sort -k2

Instances