AWS Pathways

gistfile1.md

Certification Prep: https://aws.amazon.com/certification/certification-prep/

Solutions Architect: https://aws.amazon.com/training/course-descriptions/architect/ https://aws.amazon.com/certification/certified-solutions-architect-associate/

Reading Materials:

• https://d0.awsstatic.com/whitepapers/AWS_Cloud_Best_Practices.pdf

Exam Tips

• https://acloud.guru/forums/aws-certified-solutions-architect-associate/discussion/-KKr5HCv2bzH3EOBSUAt/my_path_to_solutions_architect
• http://cantrill.io/certification/aws/2016/03/27/how-to-pass-AWS-certifications.html
• https://www.cozero.com.au/news/2017/424-things-you-should-know-about-aws.html
• https://acloud.guru/forums/aws-certified-solutions-architect-associate/discussion/-KSDNs4nfg5ikp6yBN9l/exam_feedback
• read CloudGuru forums
• https://www.linkedin.com/pulse/how-get-all-aws-certifications-asia-wong-chun-yin-cyrus-%E9%BB%83%E4%BF%8A%E5%BD%A5-/

Interesting links: Breaking the monolith: https://aws.amazon.com/getting-started/container-microservices-tutorial/ Make diagrams: https://draw.io

h3. S3 https://aws.amazon.com/s3/faqs/

• Manage Access to Your S3 Resources: http://docs.aws.amazon.com/AmazonS3/latest/dev/intro-managing-access-s3-resources.html
• Log Requests to Your Bucket: http://docs.aws.amazon.com/AmazonS3/latest/UG/ManagingBucketLogging.html To manage the lifecycle of an object, we provide the ability to automatically delete or transition objects to Glacier: http://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html To get started with event driven workflows, you can configure Amazon S3 Event Notifications: http://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html S3 also supports many use cases like running a web service/application or running enterprise applications. We have several narratives that can help you on this journey like:
• Host a Static Website: http://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteHosting.html
• Monitor Storage Metrics with Amazon CloudWatch: http://docs.aws.amazon.com/AmazonS3/latest/dev/cloudwatch-monitoring.html
• Programmatically Access S3 with the AWS Command Line Interface: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-set-up.html
• Optimize Performance for High Request Rates (More Than 100 Requests per Second): http://docs.aws.amazon.com/AmazonS3/latest/dev/request-rate-perf-considerations.html https://aws.amazon.com/answers/infrastructure-management/crr-monitor/

17/12 - Sunday

VPC Summary and exam tips

• NAT instance - disable source / destination check on the instance, it must be in public subnet. Traffic that it can handle - depends on the instance size. Create high availability using autoscaling groups, multiple subnets in different AZs and script to automate failover (hm?). It is behind a security group.
• NAT gateways - remember to update route table - may take 15 mins to provision.

19/12

Overview of security processes (part 2)

Make sure you read the security white paper - it's quite long at 95 pages though.
Encryption of data is generally a good practice, you can encrypt EBS volumes and their volumes with AES-256.
This mean data moving EC2 instances and EBS storages become secure.
However this feature only available on more powerful instances such as M3, C3, R3, G2.

ELB - SSL termination on load balancer is supported -> why is this good? this is good because then your webserver doesn't need to do decryption on their end which saves them operation power. ELB does pass the originating IP address to your webserver.

Direct Connect - bypass internet service in your network path. You can buy rack space within AWS Direct Connect location and deploy your equipment nearby.

You can conduct vulnerability scan on your instances - but you must tell Amazon beforehand - failing in doing so is a violation to your thing. AWS conduct scans on their own systems - not customers' instance.

Compliance - AWS complies to a lot of standards, one of them is PCI DSS 1 - this is compliance on the infrastructure level though, you need to make sure your app is compliance.

Storage options in cloud whitepaper

S3, Glacier, EBS, EC2 instance storage -> ephemeral (gone when you terminate the instance)
AWS Import/Export -> a service that allows you to send your data to AWS Import/Export centre - not via the internet, but by mailing or sending your physical storage. Apparently Snowball is more preferable.
AWS Storage Gateway -> connect on prem software with cloud storage. The purpose is you use cloud for data storage. It is essentially and interestingly is a VM that you install on your prem. Once installed - you can create gateway cached or gateway stored volumes that can be mounted as iSCSI devices by your on prem apps.
Gateway cached -> uses S3 for primary data - while retain some of data local in a cache for frequently access data.
you can create storage volumes up to 32TB in size.
Gateway stored ->

21/12

Kinesis Firehose vs Kinesis Streams

Kinesis Streams - You must manually provision the appropriate number of shards for your stream to handle the volume of data you expect to process. Amazon helpfully provides a shard calculator when creating a stream to correctly determine this number. Once created, it is possible to dynamically scale up or down the number of shards to meet demand, but only with the AWS Streams API at this time. with kinesis streams you do stuffs

Kinesis Firehose is Amazon’s data-ingestion product offering for Kinesis. It is used to capture and load streaming data into other Amazon services such as S3 and Redshift. From there, you can load the streams into data processing and analysis tools like Elastic Map Reduce, and Amazon Elasticsearch Service. It is also possible to load the same data into S3 and Redshift at the same time using Firehose.
In Firehose - you don't have to worry about consumer.

You can check out this re-Invent recording on Kinesis: https://www.youtube.com/watch?v=SmcgiweeviY. What I found the key difference is where you flow data through Firehose, it doesn't store. It happens under the cover for streaming data to flow through to the configured destination where you store it persistently ie S3, Redshift or ES.

22/12

OK - so I have finished all the major components in acloudguru - progress is 76% now, except for:

• ch 10: hands on lab
• ch 12: well architected frameworks
• ch 13: additional exam tips

24/12

Wordpress lab

draw.io -> website to create diagram.
The architecture of for this lab -> ELB, EC2 with auto scaling (2 instances), RDS multi AZ too (2 RDS).
The EC2s & ELB are inside a VPC with WebDMZ, RDS is inside private VPC.
IAM roles -> create a role that allow EC2 to have full access to S3
Creating security group for the EC2 -> created by going to VPC - so remember security group is part of VPC.

At this stage - I am a bit haze about EC2 auto scaling - so I am taking a detour here and re-read how to set it up

When it gets to setting up security group - I stopped the lecture and trying to remember myself how to set up a VPC that for my EC2 - the VPC should allow:

• public access to my EC2 / ELB (ingress) on port 80, 443 and 22. egress on port 80, 443 - in the case of contacting updates? I don't think it needs egress on port 22?

Steps:
Security groups and VPC

• create VPC with CIDR 10.0.0.0/16 -> wordpress VPC - so I guess VPC is per app in this case if I have to app / websites for different clients, I'd create different VPCs to isolate them.
• create a security group - web DMZ - with inbound rule:
  HTTP (80) | TCP (6) | 80 | 0.0.0.0/0 |  
  SSH (22) | TCP (6) | 22 | 0.0.0.0/0 |  
  HTTPS (443) | TCP (6) | 443 | 0.0.0.0/0 |  
• create security group for aurora - with 3306 on inbound rule - but hang on you can have multiple security groups in one VPC?
  see the inbound rule:
  MySQL/Aurora (3306) | TCP (6) | 3306 | sg-2133d255 -> the last one is the my web security group (kinda weird specifying security group as the allowable source - and we haven't touched on subnet yet). I guess what that means is allow MySQL connection from any instances that have security group of web - for instance you may have multiple EC2s with web security group, they will be allowed connection to this resource.
• created 2 subnets for wordpress VPC - specifying CIDR range a bit tricky - used both cidr.xzy and example from default VPC -> 10.0.0.0/20 and 10.0.16.0/20. What's confusing is - these subnets also have route table - I am not sure what they are for - I forgot.
• created an internet gateway and attach it to the VPC - also added the internet gateway to the route table
• this route table seems to have been created automatically?

ELB

create an ELB application one - I am not sure, should I let it listens to port 80 only?
In this ELB you need to specify which VPC - when I choose wordpress VPC no AZ is shown!!! Which means I am missing some steps here - the error message says 2 subnets must be specified - OK, so that means I need to create at least 2 subnets and obviously on 2 different AZs? added subnet above.
ok added 2 subnets - OMG - now it's complaning: You are creating an internet-facing Load Balancer, but there is no Internet Gateway attached to these subnets you have selected: subnet-5a3d7d3e, subnet-25f87f0a

Side notes:

are you sure you want to delete this vpc? - the following will be deleted too: subnets, security groups, network acls, vpn attachments, internet gateways, route tables, network interfaces, vpc peering connections.

27/12

Took few days break - Christmas and spending time with kids is kinda important.

WordPress lab - setting up EC2

Now on setting up EC2 - after I set up my EC2 - I found that there's no public IP address on the EC2 - wooottt! Turns out I need to turn on auto assign public IP address setting on my subnet!!!!

30/12

Adding resilience and cloudfront lab

Ryan backs up the wordpress code inside /var/www/html into s3 bucket. Felix note: I would probably use git for this.

aws s3 sync --delete /var/www/html/wp-content/upload s3://my-little-pony (--dry-run) -> ah pretty cool - rsync for s3 - I could use this for my blog later.

Then we do some URL rewriting magic on WP so the files are served from CloudFront instead of EC2 / WP.
Next step is to automate the sync process using cron - which is basically pasting the command above into crontab.

Setting up AMIs lab

Why bother creating ELB for one EC2 instance? It's due to the public IP address, when EC2 is re-started - it will get a new IP address.
Note to Felix: can't Cloud53 automatically connect to that EC2 based on ARN? No -> https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resource-record-sets-values-alias.html - you can only set ELB, Elastic beanstalk, cloudfront distro, S3 buckets as alias record destination.
According to Ryan: you could avoid this issue by using ELB or Elastic IP.

In this lab - Ryan split the wordpress site into 2: the production WP and the writer WP and built AMI for both of them. I don't really understand the practicality of this lab in the real world hence I didn't do it.

Autoscaling and Load Testing

didn't do ..

Exam tips based on students feedback

Kinesis - it is the way to consume big data / data stream or bringing it to the cloud - like social media
business intelligence -> RedShift
big data processing -> EMR (processing)

EC2 - EBS backed vs Instance store, know the difference. EBS store data long term.

Opswork -> orchestration services that uses Chef.

SWF Actors: workflow starters (initiate workflow), deciders (control flow), activity workers (carry out activity tasks)

AWS Organisations & consolidated billing

This is a feature for larger organisations where it may have a lot of AWS accounts. Consolidated billing is sort of replaced by AWS Organizations?
What is AWS orgs?
one root account and you can have multiple organisation units below it. Then you can have granular control for policies.

Ok - for consolidated billing. In this setup, root account is the paying account - then we have several linked accounts under it such as test, production and back office.
You will receive one bill - with break down for individual linked accounts. The accounts are independent though they cannot access resources of other accounts. Limit for consolidate billing is 20 linked accounts.
The good thing about this - you will get volume pricing discount.

Best practices -> paying account should be used for billing purpose only - don't deploy your resource there.

01/18

Happy new year! :)

Cross Account Access

Cross account access - what is it? From a post in the internet:
Today, we made it possible for you to enable a user to switch roles directly in the AWS Management Console to access resources across multiple AWS accounts—while using only one set of credentials.

I have actually experienced this at FFX - using my login, and then switch to developer role and then I was able to access devs specific resources.

Not doing the lab - but taking the idea an apply it to MEC and my own account scenario - almost got it working.

Also reading IAM documentation on AWS as well as best practice for IAM - this has solidified my understanding of user, group, policy and role.

02/18

acloud guru 88% completed - the end is near, I think I can finish this course before the holiday ends - I am stoked!

Learnt a bit about docker and docker in AWS.
ECS - Elastic Container Service is managed docker service in AWS.
ECR - Elastic Container Registry is docker image registry in AWS. AWS version of DockerHub.

I had to read additional resources to wrap my head around docker stuffs again.
docker image -> template to create a docker container (in my own words).
I am still a bit fuzzy with Task Definition, Clusters.
In my own words again - Task Definition defines how to run a docker container in AWS. It is a container configuration.
Clusters is region specific and this is a place to deploy task definitions. An ECS cluster is basically autoscaling for docker, it will provision your required number of EC2 instances to run the docker image.

ECS quick tutorial from youtube not from acloudguru https://www.youtube.com/watch?v=kQBGbmrdYO4:

• push an image to ECR
• create task definition - so here you specify the image URL from the ECR. Then you configure the container here by specifying CPU requirements etc2.
• then you create a cluster. in here you specify what EC2 instance type you want, the VPC config etc2.
• and then you create a service - uhm what? don't really get it .

When you finish this if you go to EC2 - you'll see the instance that ECS created for the container. SSH in to that box, you'll see docker installed and provisioned for you. doing docker images will show 2 images, one is the ECS agent and the other one is your image.

03/18

Did Whizlabs diagnostic exam and scored 85% (51 out of 60) - pretty stocked!
Area to improve:

• the details of things I guess for example: which DB doesn't support read replica in RDS answer Oracle
• how long can a message stays in SQS - max is 14 days default is 4 days
• autoscaling - what to do if you want to change instance type on your autoscaling group. answer: create new launch configuration and replace autoscaling's existing launch config with the new one.
• direct connect is not VPN.

29/01

Few more days before the exam

Doing a cloud guru final exam, few things to review:

• Site to site VPN vs direct connect - what's required -> You need to ensure that your application in your custom VPC can communicate back to the on-premise data center. You can do this by either using a site to site VPN or Direct Connect. It will be using an internal IP address range, so you must make sure that your internal IP addresses do not overlap.
• cname vs a record
• what is AWS WAF - what filters are available
• in auto scaling - how to determine which instance to kill?
• what services are offered by trusted advisor?
• ECS - especially with regards to permission, permission can be applied to task and the instance themselves?
• SQS - what is DelaySeconds mean?
• To establish a successful site-to-site VPN connection from your on-premise network to an AWS Virtual Private Cloud, which of the following must be configured? (Choose 3)
  You must have a VPC with Hardware VPN Access, an on-premise Customer Gateway, and a Virtual Private Gateway to make the VPN connection work.
• what is Virtual Private Gateway and Customer Gateway?

Got 72%