Created
May 13, 2015 13:59
-
-
Save pchalasani/58f008dc32205701f5d3 to your computer and use it in GitHub Desktop.
spark-sql hive read time out
This file has been truncated, but you can view the full file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Logged in as: prasadch | |
| Logs for container_1428474791204_251895_01_000001 | |
| ResourceManager | |
| RM Home | |
| NodeManager | |
| Tools | |
| SLF4J: Class path contains multiple SLF4J bindings. | |
| SLF4J: Found binding in [jar:file:/grid/6/tmp/yarn-local/usercache/prasadch/filecache/2843/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar!/org/slf4j/impl/StaticLoggerBinder.class] | |
| SLF4J: Found binding in [jar:file:/home/Releases/hadoop-2.6.0.8.1504031409-20150407-000/share/hadoop-2.6.0.8.1504031409/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] | |
| SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. | |
| SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] | |
| 15/05/13 13:49:32 INFO yarn.ApplicationMaster: Registered signal handlers for [TERM, HUP, INT] | |
| 15/05/13 13:49:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable | |
| 15/05/13 13:49:33 INFO security.JniBasedUnixGroupsNetgroupMappingWithFallback: Falling back to shell based | |
| 15/05/13 13:49:34 INFO yarn.ApplicationMaster: ApplicationAttemptId: appattempt_1428474791204_251895_000001 | |
| 15/05/13 13:49:35 WARN shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded. | |
| 15/05/13 13:49:35 INFO spark.SecurityManager: Changing view acls to: prasadch | |
| 15/05/13 13:49:35 INFO spark.SecurityManager: Changing modify acls to: prasadch | |
| 15/05/13 13:49:35 INFO spark.SecurityManager: adding secret to credentials in yarn mode | |
| 15/05/13 13:49:35 INFO spark.SecurityManager: SecurityManager: authentication enabled; ui acls enabled; users with view permissions: Set(prasadch); users with modify permissions: Set(prasadch) | |
| 15/05/13 13:49:35 INFO yarn.ApplicationMaster: Starting the user application in a separate Thread | |
| 15/05/13 13:49:35 INFO yarn.ApplicationMaster: Waiting for spark context initialization | |
| 15/05/13 13:49:35 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... | |
| 15/05/13 13:49:35 INFO spark.SparkContext: Running Spark version 1.3.1 | |
| 15/05/13 13:49:35 INFO spark.SecurityManager: Changing view acls to: prasadch | |
| 15/05/13 13:49:35 INFO spark.SecurityManager: Changing modify acls to: prasadch | |
| 15/05/13 13:49:35 INFO spark.SecurityManager: SecurityManager: authentication enabled; ui acls enabled; users with view permissions: Set(prasadch); users with modify permissions: Set(prasadch) | |
| 15/05/13 13:49:36 INFO slf4j.Slf4jLogger: Slf4jLogger started | |
| 15/05/13 13:49:36 INFO Remoting: Starting remoting | |
| 15/05/13 13:49:36 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://[email protected]:48643] | |
| 15/05/13 13:49:36 INFO util.Utils: Successfully started service 'sparkDriver' on port 48643. | |
| 15/05/13 13:49:36 INFO spark.SparkEnv: Registering MapOutputTracker | |
| 15/05/13 13:49:36 INFO spark.SparkEnv: Registering BlockManagerMaster | |
| 15/05/13 13:49:37 INFO storage.DiskBlockManager: Created local directory at /grid/0/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/blockmgr-8b05e66c-e0e3-4b22-ab64-08e26dbf7f2f | |
| 15/05/13 13:49:37 INFO storage.DiskBlockManager: Created local directory at /grid/1/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/blockmgr-dd435180-8d90-4bf7-bf0f-2ec39eb932aa | |
| 15/05/13 13:49:37 INFO storage.DiskBlockManager: Created local directory at /grid/10/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/blockmgr-2ca6882c-c3f5-4155-aee2-240c172351a7 | |
| 15/05/13 13:49:37 INFO storage.DiskBlockManager: Created local directory at /grid/11/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/blockmgr-3e1f1a09-5bc6-4266-87f5-d8da77a22c15 | |
| 15/05/13 13:49:37 INFO storage.DiskBlockManager: Created local directory at /grid/2/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/blockmgr-af48eda8-6056-4c23-9b1b-ac665e09d06d | |
| 15/05/13 13:49:37 INFO storage.DiskBlockManager: Created local directory at /grid/3/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/blockmgr-82f61e20-ae23-47fc-a953-153b8294af65 | |
| 15/05/13 13:49:37 INFO storage.DiskBlockManager: Created local directory at /grid/4/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/blockmgr-a64de771-fef9-419d-92ef-f3a6638b18dd | |
| 15/05/13 13:49:37 INFO storage.DiskBlockManager: Created local directory at /grid/5/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/blockmgr-f2af9fff-d7b4-406a-ac05-6f85b1fff66f | |
| 15/05/13 13:49:37 INFO storage.DiskBlockManager: Created local directory at /grid/6/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/blockmgr-de1be6f1-806a-4882-93fe-e518fcee6283 | |
| 15/05/13 13:49:37 INFO storage.DiskBlockManager: Created local directory at /grid/7/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/blockmgr-93fbbf88-4bc6-4c24-bfb2-c8899e572b4a | |
| 15/05/13 13:49:37 INFO storage.DiskBlockManager: Created local directory at /grid/8/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/blockmgr-852095cd-6949-4130-8d24-f4c026194d63 | |
| 15/05/13 13:49:37 INFO storage.DiskBlockManager: Created local directory at /grid/9/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/blockmgr-4da65be2-67bb-448e-9fcc-5644c5007ae7 | |
| 15/05/13 13:49:37 INFO storage.MemoryStore: MemoryStore started with capacity 2.9 GB | |
| 15/05/13 13:49:37 INFO spark.HttpFileServer: HTTP File server directory is /grid/0/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/httpd-5735bd14-dc83-43d5-a6ce-c2060ae01e52 | |
| 15/05/13 13:49:37 INFO spark.HttpServer: Starting HTTP Server | |
| 15/05/13 13:49:37 INFO server.Server: jetty-8.y.z-SNAPSHOT | |
| 15/05/13 13:49:37 INFO server.AbstractConnector: Started [email protected]:34114 | |
| 15/05/13 13:49:37 INFO util.Utils: Successfully started service 'HTTP file server' on port 34114. | |
| 15/05/13 13:49:37 INFO spark.SparkEnv: Registering OutputCommitCoordinator | |
| 15/05/13 13:49:37 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter | |
| 15/05/13 13:49:37 INFO server.Server: jetty-8.y.z-SNAPSHOT | |
| 15/05/13 13:49:37 INFO server.AbstractConnector: Started [email protected]:34376 | |
| 15/05/13 13:49:37 INFO util.Utils: Successfully started service 'SparkUI' on port 34376. | |
| 15/05/13 13:49:37 INFO ui.SparkUI: Started SparkUI at http://gsta718n15.tan.ygrid.yahoo.com:34376 | |
| 15/05/13 13:49:37 WARN spark.SparkContext: "yarn-standalone" is deprecated as of Spark 1.0. Use "yarn-cluster" instead. | |
| 15/05/13 13:49:37 INFO cluster.YarnClusterScheduler: Created YarnClusterScheduler | |
| 15/05/13 13:49:38 INFO netty.NettyBlockTransferService: Server created on 49705 | |
| 15/05/13 13:49:38 INFO storage.BlockManagerMaster: Trying to register BlockManager | |
| 15/05/13 13:49:38 INFO storage.BlockManagerMasterActor: Registering block manager gsta718n15.tan.ygrid.yahoo.com:49705 with 2.9 GB RAM, BlockManagerId(<driver>, gsta718n15.tan.ygrid.yahoo.com, 49705) | |
| 15/05/13 13:49:38 INFO storage.BlockManagerMaster: Registered BlockManager | |
| 15/05/13 13:49:38 INFO yarn.ApplicationMaster: Listen to driver: akka.tcp://[email protected]:48643/user/YarnScheduler | |
| 15/05/13 13:49:38 INFO cluster.YarnClusterSchedulerBackend: ApplicationMaster registered as Actor[akka://sparkDriver/user/YarnAM#1735083312] | |
| 15/05/13 13:49:38 INFO client.RMProxy: Connecting to ResourceManager at zaniumtan-jt1.tan.ygrid.yahoo.com/98.138.174.79:8030 | |
| 15/05/13 13:49:38 INFO yarn.YarnRMClient: Registering the ApplicationMaster | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Will request 50 executor containers, each with 10 cores and 6574 MB memory including 430 MB overhead | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:6574, vCores:10>) | |
| 15/05/13 13:49:38 INFO yarn.ApplicationMaster: Started progress reporter thread - sleep time : 5000 | |
| 15/05/13 13:49:38 INFO impl.AMRMClientImpl: Received new token for : gsta704n03.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000006 for on host gsta704n03.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta704n03.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:38 INFO yarn.YarnAllocator: Received 1 containers from YARN, launching executors on 1 of them. | |
| 15/05/13 13:49:38 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:38 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:38 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:38 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:39 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:39 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta704n03.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000006/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta704n03.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000006/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:39 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 1, --hostname, gsta704n03.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:39 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta704n03.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta733n19.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta734n10.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta61265.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta732n13.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta60647.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta642n02.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta638n16.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta644n04.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta551n27.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta559n34.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta454n27.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta456n05.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta637n09.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta456n37.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta719n06.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta353n00.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta711n18.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta551n01.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta705n05.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta454n13.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta563n19.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta711n16.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta732n17.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta715n07.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta732n11.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta648n10.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta709n12.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO impl.AMRMClientImpl: Received new token for : gsta61304.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000074 for on host gsta733n19.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta733n19.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000082 for on host gsta734n10.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:43 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta734n10.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000118 for on host gsta61265.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta61265.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000124 for on host gsta732n13.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta732n13.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000134 for on host gsta60647.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta60647.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000150 for on host gsta642n02.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:43 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta642n02.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000164 for on host gsta638n16.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta638n16.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:43 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000218 for on host gsta644n04.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta644n04.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:43 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000220 for on host gsta551n27.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta551n27.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:43 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000221 for on host gsta559n34.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta559n34.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000226 for on host gsta454n27.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta454n27.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:43 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta733n19.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000074/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta733n19.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000074/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 2, --hostname, gsta733n19.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000245 for on host gsta456n05.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta733n19.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta456n05.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta732n13.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000124/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta732n13.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000124/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 5, --hostname, gsta732n13.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:43 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:43 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta732n13.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000249 for on host gsta637n09.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta637n09.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta61265.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000118/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta61265.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000118/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 4, --hostname, gsta61265.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:43 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta61265.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta642n02.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000150/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta642n02.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000150/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 7, --hostname, gsta642n02.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta642n02.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:43 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:43 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000254 for on host gsta456n37.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta456n37.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta734n10.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000082/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta734n10.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000082/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 3, --hostname, gsta734n10.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta559n34.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000221/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta559n34.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000221/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta638n16.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000164/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta638n16.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000164/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta454n27.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000226/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta454n27.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000226/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 11, --hostname, gsta559n34.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta734n10.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000260 for on host gsta719n06.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta644n04.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000218/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta644n04.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000218/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta559n34.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 9, --hostname, gsta644n04.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta60647.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000134/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta60647.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000134/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta719n06.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta551n27.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000220/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta551n27.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000220/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 12, --hostname, gsta454n27.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 8, --hostname, gsta638n16.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta644n04.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 6, --hostname, gsta60647.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000297 for on host gsta353n00.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta454n27.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 10, --hostname, gsta551n27.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta60647.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta638n16.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta551n27.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta353n00.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000306 for on host gsta711n18.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta711n18.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000312 for on host gsta551n01.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta551n01.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000324 for on host gsta705n05.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta705n05.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000337 for on host gsta454n13.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta454n13.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta456n37.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000254/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta456n37.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000254/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 15, --hostname, gsta456n37.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta637n09.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000249/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta637n09.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000249/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta456n37.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000361 for on host gsta563n19.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 14, --hostname, gsta637n09.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta711n18.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000306/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta711n18.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000306/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 18, --hostname, gsta711n18.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta711n18.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta551n01.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000312/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta551n01.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000312/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta353n00.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000297/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta353n00.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000297/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 17, --hostname, gsta353n00.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta705n05.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000324/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta705n05.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000324/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta719n06.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000260/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta719n06.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000260/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta563n19.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 20, --hostname, gsta705n05.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 16, --hostname, gsta719n06.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta353n00.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000368 for on host gsta711n16.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 19, --hostname, gsta551n01.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta637n09.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta711n16.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta719n06.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta551n01.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta454n13.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000337/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta454n13.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000337/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 21, --hostname, gsta454n13.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta705n05.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000373 for on host gsta732n17.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta732n17.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta454n13.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta456n05.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000245/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta456n05.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000245/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 13, --hostname, gsta456n05.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta456n05.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta563n19.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000361/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta563n19.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000361/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 22, --hostname, gsta563n19.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta563n19.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000377 for on host gsta715n07.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta715n07.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000385 for on host gsta732n11.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta732n11.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000412 for on host gsta648n10.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta648n10.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000414 for on host gsta709n12.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta709n12.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000433 for on host gsta61304.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta61304.tan.ygrid.yahoo.com | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:49:44 INFO yarn.YarnAllocator: Received 28 containers from YARN, launching executors on 28 of them. | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta709n12.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000414/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta709n12.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000414/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta732n17.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000373/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta732n17.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000373/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 28, --hostname, gsta709n12.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 24, --hostname, gsta732n17.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta732n17.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta709n12.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta732n11.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000385/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta732n11.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000385/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 26, --hostname, gsta732n11.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta732n11.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta648n10.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000412/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta648n10.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000412/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 27, --hostname, gsta648n10.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta648n10.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta61304.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000433/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta61304.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000433/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 29, --hostname, gsta61304.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta61304.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta711n16.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000368/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta711n16.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000368/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 23, --hostname, gsta711n16.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta711n16.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta715n07.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000377/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta715n07.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000377/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:49:44 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 25, --hostname, gsta715n07.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:49:44 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta715n07.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:49:50 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:48717/user/Executor#-1606809923] with ID 1 | |
| 15/05/13 13:49:50 INFO storage.BlockManagerMasterActor: Registering block manager gsta704n03.tan.ygrid.yahoo.com:49601 with 3.1 GB RAM, BlockManagerId(1, gsta704n03.tan.ygrid.yahoo.com, 49601) | |
| 15/05/13 13:49:57 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:42859/user/Executor#926224354] with ID 6 | |
| 15/05/13 13:49:57 INFO storage.BlockManagerMasterActor: Registering block manager gsta60647.tan.ygrid.yahoo.com:56156 with 3.1 GB RAM, BlockManagerId(6, gsta60647.tan.ygrid.yahoo.com, 56156) | |
| 15/05/13 13:49:57 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:36750/user/Executor#-1194519243] with ID 8 | |
| 15/05/13 13:49:57 INFO storage.BlockManagerMasterActor: Registering block manager gsta638n16.tan.ygrid.yahoo.com:46412 with 3.1 GB RAM, BlockManagerId(8, gsta638n16.tan.ygrid.yahoo.com, 46412) | |
| 15/05/13 13:49:59 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:42214/user/Executor#-230527037] with ID 4 | |
| 15/05/13 13:50:00 INFO storage.BlockManagerMasterActor: Registering block manager gsta61265.tan.ygrid.yahoo.com:47790 with 3.1 GB RAM, BlockManagerId(4, gsta61265.tan.ygrid.yahoo.com, 47790) | |
| 15/05/13 13:50:01 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:54366/user/Executor#-1040810905] with ID 27 | |
| 15/05/13 13:50:01 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:46502/user/Executor#1721746292] with ID 13 | |
| 15/05/13 13:50:01 INFO storage.BlockManagerMasterActor: Registering block manager gsta456n05.tan.ygrid.yahoo.com:55548 with 3.1 GB RAM, BlockManagerId(13, gsta456n05.tan.ygrid.yahoo.com, 55548) | |
| 15/05/13 13:50:01 INFO storage.BlockManagerMasterActor: Registering block manager gsta648n10.tan.ygrid.yahoo.com:57199 with 3.1 GB RAM, BlockManagerId(27, gsta648n10.tan.ygrid.yahoo.com, 57199) | |
| 15/05/13 13:50:02 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:53285/user/Executor#504522529] with ID 23 | |
| 15/05/13 13:50:02 INFO storage.BlockManagerMasterActor: Registering block manager gsta711n16.tan.ygrid.yahoo.com:51619 with 3.1 GB RAM, BlockManagerId(23, gsta711n16.tan.ygrid.yahoo.com, 51619) | |
| 15/05/13 13:50:04 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:35147/user/Executor#1150807766] with ID 20 | |
| 15/05/13 13:50:04 INFO storage.BlockManagerMasterActor: Registering block manager gsta705n05.tan.ygrid.yahoo.com:58465 with 3.1 GB RAM, BlockManagerId(20, gsta705n05.tan.ygrid.yahoo.com, 58465) | |
| 15/05/13 13:50:04 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:38827/user/Executor#-101450617] with ID 18 | |
| 15/05/13 13:50:04 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:58077/user/Executor#-1707824079] with ID 22 | |
| 15/05/13 13:50:04 INFO storage.BlockManagerMasterActor: Registering block manager gsta711n18.tan.ygrid.yahoo.com:59807 with 3.1 GB RAM, BlockManagerId(18, gsta711n18.tan.ygrid.yahoo.com, 59807) | |
| 15/05/13 13:50:04 INFO storage.BlockManagerMasterActor: Registering block manager gsta563n19.tan.ygrid.yahoo.com:60512 with 3.1 GB RAM, BlockManagerId(22, gsta563n19.tan.ygrid.yahoo.com, 60512) | |
| 15/05/13 13:50:05 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:36436/user/Executor#1863862355] with ID 14 | |
| 15/05/13 13:50:05 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:47435/user/Executor#-192189879] with ID 12 | |
| 15/05/13 13:50:05 INFO storage.BlockManagerMasterActor: Registering block manager gsta637n09.tan.ygrid.yahoo.com:51594 with 3.1 GB RAM, BlockManagerId(14, gsta637n09.tan.ygrid.yahoo.com, 51594) | |
| 15/05/13 13:50:05 INFO storage.BlockManagerMasterActor: Registering block manager gsta454n27.tan.ygrid.yahoo.com:48966 with 3.1 GB RAM, BlockManagerId(12, gsta454n27.tan.ygrid.yahoo.com, 48966) | |
| 15/05/13 13:50:05 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:55509/user/Executor#344049812] with ID 2 | |
| 15/05/13 13:50:05 INFO storage.BlockManagerMasterActor: Registering block manager gsta733n19.tan.ygrid.yahoo.com:54300 with 3.1 GB RAM, BlockManagerId(2, gsta733n19.tan.ygrid.yahoo.com, 54300) | |
| 15/05/13 13:50:07 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:58616/user/Executor#-1507133837] with ID 17 | |
| 15/05/13 13:50:07 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:49983/user/Executor#1012617694] with ID 3 | |
| 15/05/13 13:50:07 INFO storage.BlockManagerMasterActor: Registering block manager gsta353n00.tan.ygrid.yahoo.com:37746 with 3.1 GB RAM, BlockManagerId(17, gsta353n00.tan.ygrid.yahoo.com, 37746) | |
| 15/05/13 13:50:07 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:44199/user/Executor#-1286385671] with ID 26 | |
| 15/05/13 13:50:07 INFO storage.BlockManagerMasterActor: Registering block manager gsta734n10.tan.ygrid.yahoo.com:43229 with 3.1 GB RAM, BlockManagerId(3, gsta734n10.tan.ygrid.yahoo.com, 43229) | |
| 15/05/13 13:50:07 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:49816/user/Executor#1067887990] with ID 10 | |
| 15/05/13 13:50:07 INFO storage.BlockManagerMasterActor: Registering block manager gsta732n11.tan.ygrid.yahoo.com:52419 with 3.1 GB RAM, BlockManagerId(26, gsta732n11.tan.ygrid.yahoo.com, 52419) | |
| 15/05/13 13:50:07 INFO cluster.YarnClusterSchedulerBackend: SchedulerBackend is ready for scheduling beginning after waiting maxRegisteredResourcesWaitingTime: 30000(ms) | |
| 15/05/13 13:50:07 INFO cluster.YarnClusterScheduler: YarnClusterScheduler.postStartHook done | |
| 15/05/13 13:50:08 INFO storage.BlockManagerMasterActor: Registering block manager gsta551n27.tan.ygrid.yahoo.com:39378 with 3.1 GB RAM, BlockManagerId(10, gsta551n27.tan.ygrid.yahoo.com, 39378) | |
| 15/05/13 13:50:08 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:54487/user/Executor#857633009] with ID 5 | |
| 15/05/13 13:50:08 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore. | |
| 15/05/13 13:50:08 WARN conf.HiveConf: HiveConf of name hive.metastore.local does not exist | |
| 15/05/13 13:50:08 INFO storage.BlockManagerMasterActor: Registering block manager gsta732n13.tan.ygrid.yahoo.com:33516 with 3.1 GB RAM, BlockManagerId(5, gsta732n13.tan.ygrid.yahoo.com, 33516) | |
| 15/05/13 13:50:08 INFO hive.metastore: Trying to connect to metastore with URI thrift://zaniumtan-hcat.ygrid.vip.ne1.yahoo.com:50513 | |
| 15/05/13 13:50:08 INFO hive.metastore: Connected to metastore. | |
| 15/05/13 13:50:09 INFO session.SessionState: Created local directory: /grid/3/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/container_1428474791204_251895_01_000001/tmp/prasadch | |
| 15/05/13 13:50:09 INFO session.SessionState: Created local directory: /grid/3/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/container_1428474791204_251895_01_000001/tmp/c4fbb1c4-b2af-4e54-8d32-28f8256542d3_resources | |
| 15/05/13 13:50:09 INFO session.SessionState: Created HDFS directory: /tmp/prasadch/c4fbb1c4-b2af-4e54-8d32-28f8256542d3 | |
| 15/05/13 13:50:09 INFO session.SessionState: Created local directory: /grid/3/tmp/yarn-local/usercache/prasadch/appcache/application_1428474791204_251895/container_1428474791204_251895_01_000001/tmp/prasadch/c4fbb1c4-b2af-4e54-8d32-28f8256542d3 | |
| 15/05/13 13:50:09 INFO session.SessionState: No Tez session required at this point. hive.execution.engine=mr. | |
| 15/05/13 13:50:09 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:60816/user/Executor#-657347908] with ID 19 | |
| 15/05/13 13:50:09 INFO storage.BlockManagerMasterActor: Registering block manager gsta551n01.tan.ygrid.yahoo.com:36259 with 3.1 GB RAM, BlockManagerId(19, gsta551n01.tan.ygrid.yahoo.com, 36259) | |
| 15/05/13 13:50:09 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:33275/user/Executor#389050900] with ID 16 | |
| 15/05/13 13:50:09 INFO storage.BlockManagerMasterActor: Registering block manager gsta719n06.tan.ygrid.yahoo.com:60086 with 3.1 GB RAM, BlockManagerId(16, gsta719n06.tan.ygrid.yahoo.com, 60086) | |
| 15/05/13 13:50:10 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:51623/user/Executor#-1911448799] with ID 15 | |
| 15/05/13 13:50:10 INFO storage.BlockManagerMasterActor: Registering block manager gsta456n37.tan.ygrid.yahoo.com:37510 with 3.1 GB RAM, BlockManagerId(15, gsta456n37.tan.ygrid.yahoo.com, 37510) | |
| 15/05/13 13:50:11 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:40007/user/Executor#-1811635221] with ID 25 | |
| 15/05/13 13:50:11 INFO storage.BlockManagerMasterActor: Registering block manager gsta715n07.tan.ygrid.yahoo.com:47877 with 3.1 GB RAM, BlockManagerId(25, gsta715n07.tan.ygrid.yahoo.com, 47877) | |
| 15/05/13 13:50:11 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:49115/user/Executor#239478643] with ID 28 | |
| 15/05/13 13:50:11 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore. | |
| 15/05/13 13:50:11 WARN conf.HiveConf: HiveConf of name hive.metastore.local does not exist | |
| 15/05/13 13:50:11 INFO parse.ParseDriver: Parsing command: select flurry_id, count(1) from fluorobenzene.daily_data where project_id = '233295' and mobile_app_category = 'social' and event = 'start_session' and mobile_os_name = 'android' and dt = '20141201' and data_source = 'flurry' group by flurry_id | |
| 15/05/13 13:50:11 INFO storage.BlockManagerMasterActor: Registering block manager gsta709n12.tan.ygrid.yahoo.com:36635 with 3.1 GB RAM, BlockManagerId(28, gsta709n12.tan.ygrid.yahoo.com, 36635) | |
| 15/05/13 13:50:11 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:38721/user/Executor#-83532013] with ID 24 | |
| 15/05/13 13:50:11 INFO parse.ParseDriver: Parse Completed | |
| 15/05/13 13:50:11 INFO metastore.RetryingMetaStoreClient: Reconnection status for Method: getTable is false | |
| 15/05/13 13:50:11 INFO storage.BlockManagerMasterActor: Registering block manager gsta732n17.tan.ygrid.yahoo.com:36164 with 3.1 GB RAM, BlockManagerId(24, gsta732n17.tan.ygrid.yahoo.com, 36164) | |
| 15/05/13 13:50:12 INFO metastore.RetryingMetaStoreClient: Reconnection status for Method: listPartitions is false | |
| 15/05/13 13:50:12 INFO hive.metastore: Partition limit enforced, limiting based on server side configuration | |
| 15/05/13 13:50:14 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:37779/user/Executor#1999364303] with ID 11 | |
| 15/05/13 13:50:14 INFO storage.BlockManagerMasterActor: Registering block manager gsta559n34.tan.ygrid.yahoo.com:44363 with 3.1 GB RAM, BlockManagerId(11, gsta559n34.tan.ygrid.yahoo.com, 44363) | |
| 15/05/13 13:50:14 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:48886/user/Executor#2002282845] with ID 21 | |
| 15/05/13 13:50:14 INFO storage.BlockManagerMasterActor: Registering block manager gsta454n13.tan.ygrid.yahoo.com:57056 with 3.1 GB RAM, BlockManagerId(21, gsta454n13.tan.ygrid.yahoo.com, 57056) | |
| 15/05/13 13:50:14 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:58947/user/Executor#-80516929] with ID 29 | |
| 15/05/13 13:50:14 INFO storage.BlockManagerMasterActor: Registering block manager gsta61304.tan.ygrid.yahoo.com:49813 with 3.1 GB RAM, BlockManagerId(29, gsta61304.tan.ygrid.yahoo.com, 49813) | |
| 15/05/13 13:50:15 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:38267/user/Executor#-1430881731] with ID 9 | |
| 15/05/13 13:50:15 INFO storage.BlockManagerMasterActor: Registering block manager gsta644n04.tan.ygrid.yahoo.com:57952 with 3.1 GB RAM, BlockManagerId(9, gsta644n04.tan.ygrid.yahoo.com, 57952) | |
| 15/05/13 13:50:15 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:54558/user/Executor#1483931071] with ID 7 | |
| 15/05/13 13:50:16 INFO storage.BlockManagerMasterActor: Registering block manager gsta642n02.tan.ygrid.yahoo.com:49618 with 3.1 GB RAM, BlockManagerId(7, gsta642n02.tan.ygrid.yahoo.com, 49618) | |
| 15/05/13 13:51:29 INFO impl.AMRMClientImpl: Received new token for : gsta552n21.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:51:29 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000010 for on host gsta552n21.tan.ygrid.yahoo.com | |
| 15/05/13 13:51:29 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta552n21.tan.ygrid.yahoo.com | |
| 15/05/13 13:51:29 INFO yarn.YarnAllocator: Received 1 containers from YARN, launching executors on 1 of them. | |
| 15/05/13 13:51:29 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:51:29 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:51:29 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:51:29 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:51:29 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:51:29 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta552n21.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000010/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta552n21.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000010/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:51:29 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 30, --hostname, gsta552n21.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:51:29 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta552n21.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:51:46 INFO cluster.YarnClusterSchedulerBackend: Registered executor: Actor[akka.tcp://[email protected]:52346/user/Executor#190640282] with ID 30 | |
| 15/05/13 13:51:47 INFO storage.BlockManagerMasterActor: Registering block manager gsta552n21.tan.ygrid.yahoo.com:60018 with 3.1 GB RAM, BlockManagerId(30, gsta552n21.tan.ygrid.yahoo.com, 60018) | |
| 15/05/13 13:52:19 INFO impl.AMRMClientImpl: Received new token for : gsta704n10.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:52:19 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000014 for on host gsta704n10.tan.ygrid.yahoo.com | |
| 15/05/13 13:52:19 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta704n10.tan.ygrid.yahoo.com | |
| 15/05/13 13:52:19 INFO yarn.YarnAllocator: Received 1 containers from YARN, launching executors on 1 of them. | |
| 15/05/13 13:52:19 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:52:19 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:52:19 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:52:19 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:52:19 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:52:19 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta704n10.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000014/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta704n10.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000014/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar#hive-hbase-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar#hive-hbase-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar#hive-hcatalog-core-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar#hive-hcatalog-core.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar#hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar#hive-hcatalog-hbase-storage-handler.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar#hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar#hive-hcatalog-pig-adapter.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar#hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar#hive-hcatalog-server-extensions.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar#hive-hcatalog.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar#hive-jdbc-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar#hive-jdbc.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar#hive-metastore-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar#hive-metastore.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar#hive-serde-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar#hive-serde.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar#hive-service-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar#hive-service.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar#hive-shims-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar#hive-shims-0.23-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar#hive-shims-0.23.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar#hive-shims-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar#hive-shims-common-secure-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar#hive-shims-common-secure.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar#hive-shims-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar#hive-testutils-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar#hive-webhcat-java-client-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar#hive-webhcat-java-client.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar#libfb303-0.9.0.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar#libfb303.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar#zookeeper-3.4.5.jar) | |
| 15/05/13 13:52:19 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms6144m, -Xmx6144m, -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=48643', '-Dspark.ui.port=0', '-Dspark.akka.frameSize=500', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, --executor-id, 31, --hostname, gsta704n10.tan.ygrid.yahoo.com, --cores, 1, --app-id, application_1428474791204_251895, --user-class-path, file:$PWD/__app__.jar, --user-class-path, file:$PWD/guava-11.0.2.jar, --user-class-path, file:$PWD/hcatalog-support-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hcatalog-support.jar, --user-class-path, file:$PWD/hive-ant-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-beeline-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-cli.jar, --user-class-path, file:$PWD/hive-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-common.jar, --user-class-path, file:$PWD/hive-contrib-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-exec.jar, --user-class-path, file:$PWD/hive-hbase-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hbase-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-core-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-core.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-hbase-storage-handler.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-pig-adapter.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-hcatalog-server-extensions.jar, --user-class-path, file:$PWD/hive-hcatalog.jar, --user-class-path, file:$PWD/hive-jdbc-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-jdbc.jar, --user-class-path, file:$PWD/hive-metastore-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-metastore.jar, --user-class-path, file:$PWD/hive-serde-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-serde.jar, --user-class-path, file:$PWD/hive-service-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-service.jar, --user-class-path, file:$PWD/hive-shims-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-0.23.jar, --user-class-path, file:$PWD/hive-shims-common-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-shims-common-secure.jar, --user-class-path, file:$PWD/hive-shims-common.jar, --user-class-path, file:$PWD/hive-testutils-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client-0.13.0.9.1504011355.jar, --user-class-path, file:$PWD/hive-webhcat-java-client.jar, --user-class-path, file:$PWD/libfb303-0.9.0.jar, --user-class-path, file:$PWD/libfb303.jar, --user-class-path, file:$PWD/zookeeper-3.4.5.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) | |
| 15/05/13 13:52:19 INFO impl.ContainerManagementProtocolProxy: Opening proxy : gsta704n10.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:52:24 INFO impl.AMRMClientImpl: Received new token for : gsta562n17.tan.ygrid.yahoo.com:8041 | |
| 15/05/13 13:52:24 INFO yarn.YarnAllocator: Launching container container_1428474791204_251895_01_000002 for on host gsta562n17.tan.ygrid.yahoo.com | |
| 15/05/13 13:52:24 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:48643/user/CoarseGrainedScheduler, executorHostname: gsta562n17.tan.ygrid.yahoo.com | |
| 15/05/13 13:52:24 INFO yarn.YarnAllocator: Received 1 containers from YARN, launching executors on 1 of them. | |
| 15/05/13 13:52:24 INFO yarn.ExecutorRunnable: Starting Executor Container | |
| 15/05/13 13:52:24 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 | |
| 15/05/13 13:52:24 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext | |
| 15/05/13 13:52:24 INFO yarn.ExecutorRunnable: Preparing Local resources | |
| 15/05/13 13:52:24 INFO yarn.ExecutorRunnable: Prepared Local resources Map(hive-shims-common-secure-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure-0.13.0.9.1504011355.jar" } size: 72909 timestamp: 1431524955456 type: FILE visibility: PRIVATE, hive-shims-common-secure.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-secure.jar" } size: 72909 timestamp: 1431524955532 type: FILE visibility: PRIVATE, hive-jdbc-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc-0.13.0.9.1504011355.jar" } size: 143054 timestamp: 1431524954637 type: FILE visibility: PRIVATE, hive-cli-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar" } size: 28979 timestamp: 1431524952332 type: FILE visibility: PRIVATE, hcatalog-support.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar" } size: 40042 timestamp: 1431524951788 type: FILE visibility: PRIVATE, __spark__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar" } size: 164311286 timestamp: 1431524948619 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler-0.13.0.9.1504011355.jar" } size: 344616 timestamp: 1431524954073 type: FILE visibility: PRIVATE, __app__.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar" } size: 142680214 timestamp: 1431524951639 type: FILE visibility: PRIVATE, hive-hcatalog-core-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core-0.13.0.9.1504011355.jar" } size: 477456 timestamp: 1431524953245 type: FILE visibility: PRIVATE, hive-hcatalog-core.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-core.jar" } size: 477456 timestamp: 1431524953308 type: FILE visibility: PRIVATE, hive-shims-0.23-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23-0.13.0.9.1504011355.jar" } size: 35502 timestamp: 1431524955320 type: FILE visibility: PRIVATE, hive-serde-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde-0.13.0.9.1504011355.jar" } size: 779177 timestamp: 1431524954975 type: FILE visibility: PRIVATE, hive-shims-0.23.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.23.jar" } size: 35502 timestamp: 1431524955363 type: FILE visibility: PRIVATE, hive-service.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service.jar" } size: 2023552 timestamp: 1431524955247 type: FILE visibility: PRIVATE, hive-metastore-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore-0.13.0.9.1504011355.jar" } size: 4991025 timestamp: 1431524954808 type: FILE visibility: PRIVATE, hive-contrib-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar" } size: 107386 timestamp: 1431524952556 type: FILE visibility: PRIVATE, hive-exec.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar" } size: 15621852 timestamp: 1431524953008 type: FILE visibility: PRIVATE, hive-webhcat-java-client.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client.jar" } size: 89378 timestamp: 1431524957554 type: FILE visibility: PRIVATE, hive-hcatalog.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog.jar" } size: 477456 timestamp: 1431524954583 type: FILE visibility: PRIVATE, hive-ant-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar" } size: 35886 timestamp: 1431524951830 type: FILE visibility: PRIVATE, libfb303-0.9.0.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303-0.9.0.jar" } size: 275186 timestamp: 1431524957699 type: FILE visibility: PRIVATE, hive-shims-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common.jar" } size: 34398 timestamp: 1431524955568 type: FILE visibility: PRIVATE, hive-hbase-handler-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0.9.1504011355.jar" } size: 78414 timestamp: 1431524953057 type: FILE visibility: PRIVATE, hive-shims-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-0.13.0.9.1504011355.jar" } size: 2077 timestamp: 1431524955288 type: FILE visibility: PRIVATE, hive-common.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar" } size: 179911 timestamp: 1431524952514 type: FILE visibility: PRIVATE, hive-serde.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-serde.jar" } size: 779177 timestamp: 1431524955026 type: FILE visibility: PRIVATE, hive-beeline-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar" } size: 108326 timestamp: 1431524951869 type: FILE visibility: PRIVATE, hive-jdbc.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-jdbc.jar" } size: 143054 timestamp: 1431524954690 type: FILE visibility: PRIVATE, hive-exec-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar" } size: 15621852 timestamp: 1431524952816 type: FILE visibility: PRIVATE, hive-metastore.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-metastore.jar" } size: 4991025 timestamp: 1431524954918 type: FILE visibility: PRIVATE, hcatalog-support-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar" } size: 40042 timestamp: 1431524951753 type: FILE visibility: PRIVATE, hive-testutils-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-testutils-0.13.0.9.1504011355.jar" } size: 7224 timestamp: 1431524955632 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter.jar" } size: 72978 timestamp: 1431524954380 type: FILE visibility: PRIVATE, libfb303.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/libfb303.jar" } size: 275186 timestamp: 1431524957734 type: FILE visibility: PRIVATE, hive-hbase-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler.jar" } size: 78414 timestamp: 1431524953156 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions-0.13.0.9.1504011355.jar" } size: 61832 timestamp: 1431524954416 type: FILE visibility: PRIVATE, hive-shims-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-shims-common-0.13.0.9.1504011355.jar" } size: 34398 timestamp: 1431524955418 type: FILE visibility: PRIVATE, zookeeper-3.4.5.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/zookeeper-3.4.5.jar" } size: 779974 timestamp: 1431524957774 type: FILE visibility: PRIVATE, hive-webhcat-java-client-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-webhcat-java-client-0.13.0.9.1504011355.jar" } size: 89378 timestamp: 1431524957475 type: FILE visibility: PRIVATE, hive-common-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar" } size: 179911 timestamp: 1431524952479 type: FILE visibility: PRIVATE, hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-pig-adapter-0.13.0.9.1504011355.jar" } size: 72978 timestamp: 1431524954288 type: FILE visibility: PRIVATE, guava-11.0.2.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar" } size: 1648200 timestamp: 1431524951713 type: FILE visibility: PRIVATE, hive-hcatalog-hbase-storage-handler.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-hbase-storage-handler.jar" } size: 344616 timestamp: 1431524954210 type: FILE visibility: PRIVATE, hive-hcatalog-server-extensions.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hcatalog-server-extensions.jar" } size: 61832 timestamp: 1431524954463 type: FILE visibility: PRIVATE, hive-service-0.13.0.9.1504011355.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-service-0.13.0.9.1504011355.jar" } size: 2023552 timestamp: 1431524955101 type: FILE visibility: PRIVATE, hive-cli.jar -> resource { scheme: "hdfs" host: "zaniumtan-nn1.tan.ygrid.yahoo.com" port: 8020 file: "/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar" } size: 28979 timestamp: 1431524952386 type: FILE visibility: PRIVATE) | |
| 15/05/13 13:52:24 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(SPARK_YARN_USER_ENV -> JAVA_HOME=/home/gs/java/jdk64/current, CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>./hadoop-mapreduce/conf/hadoop<CPS>./hadoop-mapreduce/share/hadoop/common/*<CPS>./hadoop-mapreduce/share/hadoop/common/lib/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/*<CPS>./hadoop-mapreduce/share/hadoop/hdfs/lib/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/*<CPS>./hadoop-mapreduce/share/hadoop/yarn/lib/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/*<CPS>./hadoop-mapreduce/share/hadoop/mapreduce/lib/*, SPARK_LOG_URL_STDERR -> http://gsta562n17.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000002/prasadch/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1428474791204_251895, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 164311286,142680214,1648200,40042,40042,35886,108326,28979,28979,179911,179911,107386,15621852,15621852,78414,78414,477456,477456,344616,344616,72978,72978,61832,61832,477456,143054,143054,4991025,4991025,779177,779177,2023552,2023552,2077,35502,35502,34398,72909,72909,34398,7224,89378,89378,275186,275186,779974, SPARK_USER -> prasadch, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE,PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1431524948619,1431524951639,1431524951713,1431524951753,1431524951788,1431524951830,1431524951869,1431524952332,1431524952386,1431524952479,1431524952514,1431524952556,1431524952816,1431524953008,1431524953057,1431524953156,1431524953245,1431524953308,1431524954073,1431524954210,1431524954288,1431524954380,1431524954416,1431524954463,1431524954583,1431524954637,1431524954690,1431524954808,1431524954918,1431524954975,1431524955026,1431524955101,1431524955247,1431524955288,1431524955320,1431524955363,1431524955418,1431524955456,1431524955532,1431524955568,1431524955632,1431524957475,1431524957554,1431524957699,1431524957734,1431524957774, JAVA_HOME -> /home/gs/java/jdk64/current, SPARK_LOG_URL_STDOUT -> http://gsta562n17.tan.ygrid.yahoo.com:8042/node/containerlogs/container_1428474791204_251895_01_000002/prasadch/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/spark-assembly-1.3.1.0-hadoop2.6.0.6.1502061521.jar#__spark__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/floura.jar#__app__.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/guava-11.0.2.jar#guava-11.0.2.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support-0.13.0.9.1504011355.jar#hcatalog-support-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hcatalog-support.jar#hcatalog-support.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-ant-0.13.0.9.1504011355.jar#hive-ant-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-beeline-0.13.0.9.1504011355.jar#hive-beeline-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli-0.13.0.9.1504011355.jar#hive-cli-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-cli.jar#hive-cli.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common-0.13.0.9.1504011355.jar#hive-common-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-common.jar#hive-common.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-contrib-0.13.0.9.1504011355.jar#hive-contrib-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec-0.13.0.9.1504011355.jar#hive-exec-0.13.0.9.1504011355.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-exec.jar#hive-exec.jar,hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/prasadch/.sparkStaging/application_1428474791204_251895/hive-hbase-handler-0.13.0 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment