Dharmesh Kakadia dharmeshkakadia
dharmeshkakadia
/ ndd-spark
Created
June 11, 2018 20:19
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2018-06-09 03:04:17 WARN KubernetesTaskSetManager:66 - Lost task 0.0 in stage 58.1 (TID 12650, 10.244.2.44, executor 2): java.lang.ExceptionInInitializerError | |
| at sun.security.ssl.SSLSessionImpl.<init>(SSLSessionImpl.java:179) | |
| at sun.security.ssl.SSLSessionImpl.<init>(SSLSessionImpl.java:147) | |
| at sun.security.ssl.SSLSessionImpl.<clinit>(SSLSessionImpl.java:79) | |
| at sun.security.ssl.SSLSocketImpl.init(SSLSocketImpl.java:604) | |
| at sun.security.ssl.SSLSocketImpl.<init>(SSLSocketImpl.java:572) | |
| at sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:110) | |
| at com.microsoft.sqlserver.jdbc.TDSChannel.enableSSL(IOBuffer.java:1675) | |
| at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1977) | |
| at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:1628) |
dharmeshkakadia
/ mnist_softman_py2.ipynb
Created
May 1, 2018 00:45
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
dharmeshkakadia
/ mnist_softmax.ipynb
Created
May 1, 2018 00:27
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
dharmeshkakadia
/ logs-from-executor-in-spark-pi-my2-1524601462109-exec-3.txt
Created
April 24, 2018 20:43
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ++ id -u | |
| + myuid=0 | |
| ++ id -g | |
| + mygid=0 | |
| + set +e | |
| ++ getent passwd 0 | |
| + uidentry=root:x:0:0:root:/root:/bin/ash | |
| + set -e | |
| + '[' -z root:x:0:0:root:/root:/bin/ash ']' | |
| + SPARK_K8S_CMD=executor |
dharmeshkakadia
/ logs-from-spark-kubernetes-driver-in-spark-pi-my2-1524601462109-driver.txt
Created
April 24, 2018 20:41
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ++ id -u | |
| + myuid=0 | |
| ++ id -g | |
| + mygid=0 | |
| + set +e | |
| ++ getent passwd 0 | |
| + uidentry=root:x:0:0:root:/root:/bin/ash | |
| + set -e | |
| + '[' -z root:x:0:0:root:/root:/bin/ash ']' | |
| + SPARK_K8S_CMD=driver |
dharmeshkakadia
/ temp.log
Created
April 19, 2018 03:55
This file has been truncated, but you can view the full file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| I0418 03:14:45.340290 19867 kubelet_node_status.go:497] Using Node Hostname from cloudprovider: "k8s-mdldata-18040746-0" | |
| I0418 03:14:55.351011 19867 kubelet_node_status.go:497] Using Node Hostname from cloudprovider: "k8s-mdldata-18040746-0" | |
| I0418 03:15:05.361161 19867 kubelet_node_status.go:497] Using Node Hostname from cloudprovider: "k8s-mdldata-18040746-0" | |
| I0418 03:15:15.371310 19867 kubelet_node_status.go:497] Using Node Hostname from cloudprovider: "k8s-mdldata-18040746-0" | |
| I0418 03:15:25.381692 19867 kubelet_node_status.go:497] Using Node Hostname from cloudprovider: "k8s-mdldata-18040746-0" | |
| I0418 03:15:29.530174 19867 container_manager_linux.go:425] [ContainerManager]: Discovered runtime cgroups name: /system.slice/docker.service | |
| I0418 03:15:35.391704 19867 kubelet_node_status.go:497] Using Node Hostname from cloudprovider: "k8s-mdldata-18040746-0" | |
| I0418 03:15:45.402104 19867 kubelet_node_status.go:497] Using Node Hostname from cloudprovider: "k8s-mdldata-18040746-0" | |
| I0418 03:15:55.4125 |
dharmeshkakadia
/ spark-k8s-docker-error.sh
Created
April 16, 2018 22:57
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Dharmeshs-Mac:spark-2.3.0-bin-hadoop2.7 dharmesh$ ./bin/docker-image-tool.sh -t latest -r mdldata.azurecr.io/data/spark build | |
| + '[' -z '' ']' | |
| +++ dirname ./bin/docker-image-tool.sh | |
| ++ cd ./bin/.. | |
| ++ pwd | |
| + SPARK_HOME=/Users/dharmesh/lambda/spark-2.3.0-bin-hadoop2.7 | |
| + . /Users/dharmesh/lambda/spark-2.3.0-bin-hadoop2.7/bin/load-spark-env.sh | |
| ++ '[' -z /Users/dharmesh/lambda/spark-2.3.0-bin-hadoop2.7 ']' | |
| ++ '[' -z '' ']' | |
| ++ export SPARK_ENV_LOADED=1 |
dharmeshkakadia
/ sql-k8s-job.yaml
Last active
March 23, 2018 23:10
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| apiVersion: batch/v1 | |
| kind: Job | |
| metadata: | |
| name: sqljob | |
| spec: | |
| template: | |
| spec: | |
| containers: | |
| - name: sqljobcontainer | |
| image: microsoft/mssql-tools |
dharmeshkakadia
/ securehadoopadls.json
Created
January 27, 2018 01:44
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", | |
| "contentVersion": "1.0.0.0", | |
| "parameters": { | |
| "location": { | |
| "type": "string", | |
| "allowedValues": ["North Central US", | |
| "Central US", | |
| "East Asia", | |
| "East US", |
dharmeshkakadia
/ tpcds-counts.sql
Created
January 16, 2018 22:12
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| select count(*) from call_center; | |
| select count(*) from catalog_page; | |
| select count(*) from catalog_returns; | |
| select count(*) from catalog_sales; | |
| select count(*) from customer; | |
| select count(*) from customer_address; | |
| select count(*) from customer_demographics; | |
| select count(*) from date_dim; | |
| select count(*) from household_demographics; | |
| select count(*) from income_band; |
NewerOlder