Skip to content

Instantly share code, notes, and snippets.

@cicorias
Created December 10, 2019 14:31
Show Gist options
  • Save cicorias/db0c0c1fcea24b30c042c74eea276b46 to your computer and use it in GitHub Desktop.
Save cicorias/db0c0c1fcea24b30c042c74eea276b46 to your computer and use it in GitHub Desktop.
HDInsight creation terraform wrong vm sku azure
error with tf hdinsight
azurerm_hdinsight_hadoop_cluster.hdpcluster: Still destroying... [id=/subscriptions/b00a4572-2115-4d17-a2e5-....HDInsight/clusters/scicoriahdicluster, 40s elapsed]
azurerm_hdinsight_hadoop_cluster.hdpcluster: Still destroying... [id=/subscriptions/b00a4572-2115-4d17-a2e5-....HDInsight/clusters/scicoriahdicluster, 50s elapsed]
azurerm_hdinsight_hadoop_cluster.hdpcluster: Still destroying... [id=/subscriptions/b00a4572-2115-4d17-a2e5-....HDInsight/clusters/scicoriahdicluster, 1m0s elapsed]
azurerm_hdinsight_hadoop_cluster.hdpcluster: Destruction complete after 1m2s
Destroy complete! Resources: 1 destroyed.
data.azurerm_resource_group.hdpcluster: Refreshing state...
data.azurerm_storage_account.hdpcluster: Refreshing state...
azurerm_hdinsight_hadoop_cluster.hdpcluster: Creating...
azurerm_hdinsight_hadoop_cluster.hdpcluster: Still creating... [10s elapsed]
azurerm_hdinsight_hadoop_cluster.hdpcluster: Still creating... [30s elapsed]
azurerm_hdinsight_hadoop_cluster.hdpcluster: Still creating... [40s elapsed]
azurerm_hdinsight_hadoop_cluster.hdpcluster: Still creating... [50s elapsed]
azurerm_hdinsight_hadoop_cluster.hdpcluster: Still creating... [1m0s elapsed]
Error: Error waiting for creation of HDInsight Hadoop Cluster "scicoriahdicluster" (Resource Group "scicoria-hdp-cluster"): Code="InvalidDocumentErrorCode" Message="DeploymentDocument 'CsmDocument_2_0' failed the validation. Error: 'VM size 'standard_a2_v2' provided in the request is invalid or not supported for role 'workernode'. Valid values are: A6,A7,EXTRALARGE,LARGE,STANDARD_D12,STANDARD_D12_V2,STANDARD_D13,STANDARD_D13_V2,STANDARD_D14,STANDARD_D14_V2,STANDARD_D3,STANDARD_D3_V2,STANDARD_D4,STANDARD_D4_V2,STANDARD_D5_V2,STANDARD_DS12_V2,STANDARD_DS13_V2,STANDARD_DS14_V2,STANDARD_DS3_V2,STANDARD_DS4_V2,STANDARD_DS5_V2,STANDARD_A4_V2,STANDARD_A8_V2,STANDARD_A2M_V2,STANDARD_A4M_V2,STANDARD_A8M_V2,STANDARD_E16_V3,STANDARD_E2_V3,STANDARD_E20_V3,STANDARD_E32_V3,STANDARD_E4_V3,STANDARD_E64_V3,STANDARD_E64I_V3,STANDARD_E8_V3,STANDARD_A6,STANDARD_A7,STANDARD_F16,STANDARD_F4,STANDARD_F8'"
12: resource "azurerm_hdinsight_hadoop_cluster" "hdpcluster" {
C:\g\njit\njit-bd-mr-project\deploy\az [master ≡]> terraform apply -auto-approve
data.azurerm_resource_group.hdpcluster: Refreshing state...
data.azurerm_storage_account.hdpcluster: Refreshing state...
azurerm_hdinsight_hadoop_cluster.hdpcluster: Creating...
Error: Error creating HDInsight Hadoop Cluster "scicoriahdicluster" (Resource Group "scicoria-hdp-cluster"): hdinsight.ClustersClient#Create: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status=<nil> <nil>
12: resource "azurerm_hdinsight_hadoop_cluster" "hdpcluster" {
C:\g\njit\njit-bd-mr-project\deploy\az [master ≡]> terraform apply -auto-approve
data.azurerm_resource_group.hdpcluster: Refreshing state...
data.azurerm_storage_account.hdpcluster: Refreshing state...
azurerm_hdinsight_hadoop_cluster.hdpcluster: Creating...
Error: Error creating HDInsight Hadoop Cluster "scicoriahdicluster" (Resource Group "scicoria-hdp-cluster"): hdinsight.ClustersClient#Create: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status=<nil> <nil>
12: resource "azurerm_hdinsight_hadoop_cluster" "hdpcluster" {
C:\g\njit\njit-bd-mr-project\deploy\az [master ≡]> terraform destroy -auto-approve
data.azurerm_resource_group.hdpcluster: Refreshing state...
data.azurerm_storage_account.hdpcluster: Refreshing state...
Destroy complete! Resources: 0 destroyed.
C:\g\njit\njit-bd-mr-project\deploy\az [master ≡]> terraform apply -auto-approve
data.azurerm_resource_group.hdpcluster: Refreshing state...
data.azurerm_storage_account.hdpcluster: Refreshing state...
azurerm_hdinsight_hadoop_cluster.hdpcluster: Creating...
Error: Error creating HDInsight Hadoop Cluster "scicoriahdicluster" (Resource Group "scicoria-hdp-cluster"): hdinsight.ClustersClient#Create: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status=<nil> <nil>
on main.tf line 12, in resource "azurerm_hdinsight_hadoop_cluster" "hdpcluster":
12: resource "azurerm_hdinsight_hadoop_cluster" "hdpcluster" {
data.azurerm_storage_account.hdpcluster: Refreshing state...
azurerm_hdinsight_hadoop_cluster.hdpcluster: Creating...
Error: Error creating HDInsight Hadoop Cluster "scicoriahdicluster" (Resource Group "scicoria-hdp-cluster"): hdinsight.ClustersClient#Create: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status=<nil> <nil>
on main.tf line 12, in resource "azurerm_hdinsight_hadoop_cluster" "hdpcluster":
12: resource "azurerm_hdinsight_hadoop_cluster" "hdpcluster" {
C:\g\njit\njit-bd-mr-project\deploy\az [master ≡]> terraform apply -auto-approve
data.azurerm_resource_group.hdpcluster: Refreshing state...
data.azurerm_storage_account.hdpcluster: Refreshing state...
azurerm_hdinsight_hadoop_cluster.hdpcluster: Creating...
Error: Error creating HDInsight Hadoop Cluster "scicoriahdicluster" (Resource Group "scicoria-hdp-cluster"): hdinsight.ClustersClient#Create: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status=<nil> <nil>
on main.tf line 12, in resource "azurerm_hdinsight_hadoop_cluster" "hdpcluster":
12: resource "azurerm_hdinsight_hadoop_cluster" "hdpcluster" {
C:\g\njit\njit-bd-mr-project\deploy\az [master ≡]> dir
Directory: C:\g\njit\njit-bd-mr-project\deploy\az
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 12/9/2019 6:34 PM .terraform
-a---- 12/9/2019 6:31 PM 58 .envrc.sample
-a---- 12/9/2019 6:31 PM 718 .gitignore
-a---- 12/9/2019 6:31 PM 980 example_var_override.tf
-a---- 12/9/2019 6:31 PM 1751 main.tf
-a---- 12/9/2019 6:31 PM 667 output.tf
-a---- 12/9/2019 6:31 PM 95 providers.tf
-a---- 12/9/2019 6:31 PM 814 README.md
-a---- 12/10/2019 9:27 AM 4690 terraform.tfstate
-a---- 12/10/2019 9:26 AM 4330 terraform.tfstate.backup
-a---- 12/10/2019 9:26 AM 873 var_override.tf
-a---- 12/9/2019 6:31 PM 777 variables.tf
-a---- 12/9/2019 6:31 PM 472 versions.tf
C:\g\njit\njit-bd-mr-project\deploy\az [master ≡]> rm .\terraform.tfstate*
C:\g\njit\njit-bd-mr-project\deploy\az [master ≡]> terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.azurerm_resource_group.hdpcluster: Refreshing state...
data.azurerm_storage_account.hdpcluster: Refreshing state...
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# azurerm_hdinsight_hadoop_cluster.hdpcluster will be created
+ resource "azurerm_hdinsight_hadoop_cluster" "hdpcluster" {
+ cluster_version = "3.6"
+ https_endpoint = (known after apply)
+ id = (known after apply)
+ location = "eastus"
+ name = "scicoriahdicluster"
+ resource_group_name = "scicoria-hdp-cluster"
+ ssh_endpoint = (known after apply)
+ tags = (known after apply)
+ tier = "Standard"
+ component_version {
+ hadoop = "2.7"
+ gateway {
+ enabled = true
+ password = (sensitive value)
+ username = "cicorias"
}
+ roles {
+ head_node {
+ password = (sensitive value)
+ vm_size = "STANDARD_A4_V2"
}
+ worker_node {
+ password = (sensitive value)
+ target_instance_count = 2
+ username = "cicorias"
+ vm_size = "STANDARD_A4_V2"
}
+ zookeeper_node {
+ password = (sensitive value)
+ username = "cicorias"
+ vm_size = "STANDARD_A4_V2"
}
}
+ storage_account {
+ is_default = true
+ storage_container_id = "https://scicoriahdinsightstor.blob.core.windows.net/hdinsight"
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------
Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.
C:\g\njit\njit-bd-mr-project\deploy\az [master ≡]> terraform apply -auto-approve
data.azurerm_resource_group.hdpcluster: Refreshing state...
data.azurerm_storage_account.hdpcluster: Refreshing state...
azurerm_hdinsight_hadoop_cluster.hdpcluster: Creating...
Error: Error creating HDInsight Hadoop Cluster "scicoriahdicluster" (Resource Group "scicoria-hdp-cluster"): hdinsight.ClustersClient#Create: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status=<nil> <nil>
on main.tf line 12, in resource "azurerm_hdinsight_hadoop_cluster" "hdpcluster":
12: resource "azurerm_hdinsight_hadoop_cluster" "hdpcluster" {
C:\g\njit\njit-bd-mr-project\deploy\az [master ≡]> terraform refresh
data.azurerm_resource_group.hdpcluster: Refreshing state...
data.azurerm_storage_account.hdpcluster: Refreshing state...
Outputs:
clusterDnsName = scicoriahdicluster
sshCommand = ssh [email protected]
sshTunnelCommand = ssh -C2qTnNf -D 9876 [email protected]
storageAccountName = scicoriahdinsightstor
C:\g\njit\njit-bd-mr-project\deploy\az [master ≡]> terraform refresh
data.azurerm_resource_group.hdpcluster: Refreshing state...
data.azurerm_storage_account.hdpcluster: Refreshing state...
Outputs:
clusterDnsName = scicoriahdicluster
sshCommand = ssh [email protected]
sshTunnelCommand = ssh -C2qTnNf -D 9876 [email protected]
storageAccountName = scicoriahdinsightstor
C:\g\njit\njit-bd-mr-project\deploy\az [master ≡]> terraform apply -auto-approve
data.azurerm_resource_group.hdpcluster: Refreshing state...
data.azurerm_storage_account.hdpcluster: Refreshing state...
azurerm_hdinsight_hadoop_cluster.hdpcluster: Creating...
azurerm_hdinsight_hadoop_cluster.hdpcluster: Still creating... [10s elapsed]
azurerm_hdinsight_hadoop_cluster.hdpcluster: Still creating... [20s elapsed]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment