Created
June 21, 2018 09:12
-
-
Save gordonclarkok/3c30963ad07f5f38282d76e2371a23b1 to your computer and use it in GitHub Desktop.
My example AWS EMR cluster in cloudformation - security groups in separate file
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| AWSTemplateFormatVersion: 2010-09-09 | |
| Description: EMR Cluster with Hadoop, Hbase, Spark, Ganglia and Zookeeper | |
| Parameters: | |
| EMRClusterName: | |
| Description: Name of the cluster | |
| Type: String | |
| Default: emrcluster | |
| KeyName: | |
| Description: Must be an existing Keyname | |
| Type: String | |
| Default: mykey | |
| MasterInstanceType: | |
| Description: Instance type to be used for the master instance. | |
| Type: String | |
| Default: m4.large | |
| CoreInstanceType: | |
| Description: Instance type to be used for core instances. | |
| Type: String | |
| Default: m4.large | |
| NumberOfCoreInstances: | |
| Description: Must be a valid number | |
| Type: Number | |
| Default: 2 | |
| LogUri: | |
| Description: Must be a valid S3 URL | |
| Default: 's3://my-emr-logbucket/' | |
| Type: String | |
| S3DataUri: | |
| Description: 'Must be a valid S3 bucket URL ' | |
| Default: 's3://my-emr/' | |
| Type: String | |
| ReleaseLabel: | |
| Description: Must be a valid EMR release version | |
| Default: emr-5.11.0 | |
| Type: String | |
| Resources: | |
| EMRCluster: | |
| DependsOn: | |
| - EMRClusterServiceRole | |
| - EMRClusterinstanceProfileRole | |
| - EMRClusterinstanceProfile | |
| Type: 'AWS::EMR::Cluster' | |
| Properties: | |
| Applications: | |
| - Name: Spark | |
| - Name: Hbase | |
| - Name: Hadoop | |
| - Name: Ganglia | |
| - Name: Zookeeper | |
| Configurations: | |
| - Classification: hbase-site | |
| ConfigurationProperties: | |
| hbase.rootdir: !Ref S3DataUri | |
| - Classification: hbase | |
| ConfigurationProperties: | |
| hbase.emr.storageMode: s3 | |
| Instances: | |
| Ec2KeyName: !Ref KeyName | |
| Ec2SubnetId: !ImportValue MySubnetPrivateA | |
| EmrManagedMasterSecurityGroup: !ImportValue EmrMasterSgId | |
| AdditionalMasterSecurityGroups: | |
| - !ImportValue EmrMasterAdditionalSgId | |
| EmrManagedSlaveSecurityGroup: !ImportValue EmrSlaveSgId | |
| AdditionalSlaveSecurityGroups: | |
| - !ImportValue EmrSlaveAdditionalSgId | |
| ServiceAccessSecurityGroup: !ImportValue EmrServiceSgId | |
| MasterInstanceGroup: | |
| InstanceCount: 1 | |
| InstanceType: !Ref MasterInstanceType | |
| Market: ON_DEMAND | |
| Name: Master | |
| CoreInstanceGroup: | |
| InstanceCount: !Ref NumberOfCoreInstances | |
| InstanceType: !Ref CoreInstanceType | |
| Market: ON_DEMAND | |
| Name: Core | |
| TerminationProtected: false | |
| VisibleToAllUsers: true | |
| JobFlowRole: !Ref EMRClusterinstanceProfile | |
| ReleaseLabel: !Ref ReleaseLabel | |
| LogUri: !Ref LogUri | |
| Name: !Ref EMRClusterName | |
| AutoScalingRole: EMR_AutoScaling_DefaultRole | |
| ServiceRole: !Ref EMRClusterServiceRole | |
| Tags: | |
| - | |
| Key: "cluster_name" | |
| Value: "master.emr.my.com" | |
| EMRClusterServiceRole: | |
| Type: 'AWS::IAM::Role' | |
| Properties: | |
| AssumeRolePolicyDocument: | |
| Version: 2012-10-17 | |
| Statement: | |
| - Effect: Allow | |
| Principal: | |
| Service: | |
| - elasticmapreduce.amazonaws.com | |
| Action: | |
| - 'sts:AssumeRole' | |
| ManagedPolicyArns: | |
| - 'arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceRole' | |
| Path: / | |
| EMRClusterinstanceProfileRole: | |
| Type: 'AWS::IAM::Role' | |
| Properties: | |
| AssumeRolePolicyDocument: | |
| Version: 2012-10-17 | |
| Statement: | |
| - Effect: Allow | |
| Principal: | |
| Service: | |
| - ec2.amazonaws.com | |
| Action: | |
| - 'sts:AssumeRole' | |
| ManagedPolicyArns: | |
| - 'arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceforEC2Role' | |
| Path: / | |
| EMRClusterinstanceProfile: | |
| Type: 'AWS::IAM::InstanceProfile' | |
| Properties: | |
| Path: / | |
| Roles: | |
| - !Ref EMRClusterinstanceProfileRole | |
| Outputs: {} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment