Created
February 9, 2017 12:00
-
-
Save nsphung/91dd5c0509f534d59345f10d6969d54f to your computer and use it in GitHub Desktop.
dataproc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ gcloud dataproc clusters diagnose fca-cluster-202 [12:47:56] | |
| Waiting on operation [projects/fcms-sandbox-gce/regions/global/operations/cddd33c1-1a4a-4e23-8309-54e0fcfb4063]. | |
| Waiting for cluster diagnose operation...done. | |
| Output from diagnostic: | |
| ----------------------------------------------- | |
| Running Google Cloud Dataproc diagnostic | |
| Collecting daemon and system state information | |
| 17/02/09 11:48:14 INFO client.RMProxy: Connecting to ResourceManager at fca-cluster-202-m/10.132.0.30:8032 | |
| 17/02/09 11:48:16 INFO gcs.GoogleHadoopFileSystemBase: GHFS version: 1.5.5-hadoop2 | |
| ping: unknown host fca-cluster-202-sw-3por.c.fcms-sandbox-gce.internal | |
| ping: unknown host fca-cluster-202-sw-v7fz.c.fcms-sandbox-gce.internal | |
| ping: unknown host fca-cluster-202-w-6.c.fcms-sandbox-gce.internal | |
| Collecting daemon logs | |
| Collecting configs | |
| Collecting node startup logs | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-m/dataproc-startup-script_SUCCESS... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-m/dataproc-initialization-script-0_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-m/dataproc-startup-script_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-sw-3por/dataproc-initialization-script-0_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-sw-3por/dataproc-startup-script_SUCCESS... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-sw-3por/dataproc-startup-script_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-sw-v7fz/dataproc-initialization-script-0_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-sw-v7fz/dataproc-startup-script_SUCCESS... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-0/dataproc-startup-script_SUCCESS... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-0/dataproc-initialization-script-0_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-sw-v7fz/dataproc-startup-script_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-0/dataproc-startup-script_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-1/dataproc-initialization-script-0_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-1/dataproc-startup-script_SUCCESS... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-1/dataproc-startup-script_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-2/dataproc-initialization-script-0_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-2/dataproc-startup-script_SUCCESS... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-2/dataproc-startup-script_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-3/dataproc-initialization-script-0_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-3/dataproc-startup-script_SUCCESS... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-3/dataproc-startup-script_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-4/dataproc-startup-script_SUCCESS... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-4/dataproc-initialization-script-0_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-5/dataproc-startup-script_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-4/dataproc-startup-script_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-5/dataproc-initialization-script-0_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-6/dataproc-startup-script_SUCCESS... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-5/dataproc-startup-script_SUCCESS... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-6/dataproc-startup-script_output... | |
| Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-6/dataproc-initialization-script-0_output... | |
| / [30/30 files][733.9 KiB/733.9 KiB] 100% Done | |
| Operation completed over 30 objects/733.9 KiB. | |
| Collecting stack traces | |
| 22719: Unable to open socket file: target process not responding or HotSpot VM not loaded | |
| The -F option can be used when the target process is not responding | |
| Generating archive | |
| hadoop_conf/ | |
| hadoop_conf/mapred-queues.xml.template | |
| hadoop_conf/hadoop-policy.xml | |
| hadoop_conf/capacity-scheduler.xml | |
| hadoop_conf/configuration.xsl | |
| hadoop_conf/hadoop-env.sh | |
| hadoop_conf/mapred-site.xml | |
| hadoop_conf/hdfs-site.xml | |
| hadoop_conf/mapred-env.sh | |
| hadoop_conf/yarn-env.sh | |
| hadoop_conf/ssl-client.xml.example | |
| hadoop_conf/log4j.properties | |
| hadoop_conf/slaves | |
| hadoop_conf/mapred-site.xml.template | |
| hadoop_conf/nodes_exclude | |
| hadoop_conf/hadoop-metrics2.properties | |
| hadoop_conf/ssl-server.xml.example | |
| hadoop_conf/hadoop-metrics.properties | |
| hadoop_conf/distcp-default.xml | |
| hadoop_conf/container-executor.cfg | |
| hadoop_conf/yarn-site.xml | |
| hadoop_conf/nodes_include | |
| hadoop_conf/core-site.xml | |
| jstack/ | |
| jstack/driver_22719.jstack | |
| jstack/agent.jstack | |
| logs/ | |
| logs/hive-metastore.log | |
| logs/hadoop-hdfs-secondarynamenode-fca-cluster-202-m.log | |
| logs/google-dataproc-agent.0.log | |
| logs/hadoop-hdfs-namenode-fca-cluster-202-m.log | |
| logs/yarn-yarn-resourcemanager-fca-cluster-202-m.log | |
| logs/mapred-mapred-historyserver-fca-cluster-202-m.log | |
| logs/hive-server2.log | |
| logs/syslog | |
| node_startup/ | |
| node_startup/fca-cluster-202-sw-3por/ | |
| node_startup/fca-cluster-202-sw-3por/dataproc-startup-script_SUCCESS | |
| node_startup/fca-cluster-202-sw-3por/dataproc-initialization-script-0_output | |
| node_startup/fca-cluster-202-sw-3por/dataproc-startup-script_output | |
| node_startup/fca-cluster-202-w-3/ | |
| node_startup/fca-cluster-202-w-3/dataproc-startup-script_SUCCESS | |
| node_startup/fca-cluster-202-w-3/dataproc-initialization-script-0_output | |
| node_startup/fca-cluster-202-w-3/dataproc-startup-script_output | |
| node_startup/fca-cluster-202-w-4/ | |
| node_startup/fca-cluster-202-w-4/dataproc-startup-script_SUCCESS | |
| node_startup/fca-cluster-202-w-4/dataproc-initialization-script-0_output | |
| node_startup/fca-cluster-202-w-4/dataproc-startup-script_output | |
| node_startup/fca-cluster-202-w-0/ | |
| node_startup/fca-cluster-202-w-0/dataproc-startup-script_SUCCESS | |
| node_startup/fca-cluster-202-w-0/dataproc-initialization-script-0_output | |
| node_startup/fca-cluster-202-w-0/dataproc-startup-script_output | |
| node_startup/fca-cluster-202-sw-v7fz/ | |
| node_startup/fca-cluster-202-sw-v7fz/dataproc-startup-script_SUCCESS | |
| node_startup/fca-cluster-202-sw-v7fz/dataproc-initialization-script-0_output | |
| node_startup/fca-cluster-202-sw-v7fz/dataproc-startup-script_output | |
| node_startup/fca-cluster-202-w-2/ | |
| node_startup/fca-cluster-202-w-2/dataproc-startup-script_SUCCESS | |
| node_startup/fca-cluster-202-w-2/dataproc-initialization-script-0_output | |
| node_startup/fca-cluster-202-w-2/dataproc-startup-script_output | |
| node_startup/fca-cluster-202-m/ | |
| node_startup/fca-cluster-202-m/dataproc-startup-script_SUCCESS | |
| node_startup/fca-cluster-202-m/dataproc-initialization-script-0_output | |
| node_startup/fca-cluster-202-m/dataproc-startup-script_output | |
| node_startup/fca-cluster-202-w-6/ | |
| node_startup/fca-cluster-202-w-6/dataproc-startup-script_SUCCESS | |
| node_startup/fca-cluster-202-w-6/dataproc-initialization-script-0_output | |
| node_startup/fca-cluster-202-w-6/dataproc-startup-script_output | |
| node_startup/fca-cluster-202-w-1/ | |
| node_startup/fca-cluster-202-w-1/dataproc-startup-script_SUCCESS | |
| node_startup/fca-cluster-202-w-1/dataproc-initialization-script-0_output | |
| node_startup/fca-cluster-202-w-1/dataproc-startup-script_output | |
| node_startup/fca-cluster-202-w-5/ | |
| node_startup/fca-cluster-202-w-5/dataproc-startup-script_SUCCESS | |
| node_startup/fca-cluster-202-w-5/dataproc-initialization-script-0_output | |
| node_startup/fca-cluster-202-w-5/dataproc-startup-script_output | |
| system/ | |
| system/cluster-ping.log | |
| system/df.log | |
| system/netstat.log | |
| system/fs-file-nr.log | |
| system/yarn-nodes.log | |
| system/sysctl.log | |
| system/free.log | |
| system/hdfs-nodes.log | |
| system/ps.log | |
| system/service.log | |
| system/uptime.log | |
| Saving archive to cloud | |
| Copying file:///tmp/tmp.7raWGkmARy/diagnostic.tar [Content-Type=application/x-tar]... | |
| ==> NOTE: You are uploading one or more large file(s), which would run | |
| significantly faster if you enable parallel composite uploads. This | |
| feature can be enabled by editing the | |
| "parallel_composite_upload_threshold" value in your .boto | |
| configuration file. However, note that if you do this large files will | |
| be uploaded as `composite objects | |
| <https://cloud.google.com/storage/docs/composite-objects>`_,which | |
| means that any user who downloads such objects will need to have a | |
| compiled crcmod installed (see "gsutil help crcmod"). This is because | |
| without a compiled crcmod, computing checksums on composite objects is | |
| so slow that gsutil disables downloads of composite objects. | |
| / [1 files][240.9 MiB/240.9 MiB] | |
| Operation completed over 1 objects/240.9 MiB. | |
| Diagnostic results saved in: gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/tasks/task-cddd33c1-1a4a-4e23-8309-54e0fcfb4063/diagnostic.tar | |
| If diagnostic bundle will be shared with Google Support, then | |
| you can grant read permission to Google Support by running: | |
| gsutil acl ch -p viewers-24135231894:R gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/tasks/task-cddd33c1-1a4a-4e23-8309-54e0fcfb4063/diagnostic.tar | |
| ----------------------------------------------- | |
| gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/tasks/task-cddd33c1-1a4a-4e23-8309-54e0fcfb4063/output | |
| ... | |
| Updates are available for some Cloud SDK components. To install them, | |
| please run: | |
| $ gcloud components update |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment