Neeraj nsabharwal
nsabharwal
/ nginx
Created
April 7, 2016 19:48
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| boot2docker shellinit | |
| docker run -d -p 8080:80 --name web nginx | |
| docker exec -it web bash | |
| echo myself for president > /usr/share/nginx/html/index.html | |
| docker inspect web | grep IP | |
| docker inspect -f '{{json .NetworkSettings.IPAddress}}' container |
nsabharwal
/ Docker test
Last active
April 7, 2016 19:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ##Create the vm | |
| boot2docker init -m 8192 -s 30000 ( memory and size in GB) | |
| ###Start the vm | |
| boot2docker up | |
| # Set the variables | |
| `eval "$(boot2docker shellinit)"` | |
| ## Find the IP | |
| boot2docker IP | |
| ##ssh into docker vm | |
| boot2docker ssh |
nsabharwal
/ Kafka validation
Created
February 15, 2016 14:36
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --list --zookeeper localhost:2181 | |
| /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper localhost:2181 --create --topic test --replication-factor 1 --partitions 1 | |
| /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list phdns01.cloud.hortonworks.com:6667 --topic test | |
| /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning |
nsabharwal
/ Zeppelin and SparkR
Last active
March 6, 2017 01:44
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #Install R and then following packages | |
| #repr failed to create | |
| yum install R-* | |
| install.packages("evaluate", dependencies = TRUE) | |
| install.packages("base64enc", dependencies = TRUE) | |
| install.packages("devtools", dependencies = TRUE) | |
| install_github('IRkernel/repr') | |
| install.packages("dplyr", dependencies = TRUE) | |
| install.packages("caret", dependencies = TRUE) | |
| install.packages("repr", dependencies = TRUE) |
nsabharwal
/ Google Cloud Storage Test
Created
December 13, 2015 17:53
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| HDFS test | |
| Make sure that google connector is defined in Hadoop CLASSPATH as decribed in the blog | |
| [hdfs@hdpgcp-1-1435537523061 ~]$ hdfs dfs -ls gs://hivetest/ | |
| 15/06/28 21:15:32 INFO gcs.GoogleHadoopFileSystemBase: GHFS version: 1.4.0-hadoop2 | |
| 15/06/28 21:15:33 WARN gcs.GoogleHadoopFileSystemBase: No working directory configured, using default: 'gs://hivetest/' |
nsabharwal
/ hadoopcleanup
Created
December 10, 2015 12:04
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ambari-server stop | |
| ambari-agent stop | |
| pkill -9 java | |
| ################################# | |
| # Remove Packages | |
| ################################ | |
| yum -y remove ambari-\* | |
| yum -y remove hcatalog\* | |
| yum -y remove hive\* |
nsabharwal
/ Hive DDL
Created
November 21, 2015 03:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| mysql -u hive -p -e " select concat( 'show create table ' , TBL_NAME,';') from TBLS" hive > file.sql | |
| hive -f /tmp/file.sql |
nsabharwal
/ beeline script
Created
November 11, 2015 16:48
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| read -p "enter HS2 hostname: " HS2 | |
| read -p "enter username: " username | |
| echo "enter password" | |
| read -s passwd | |
| read -p "enter filename: " filename | |
| beeline -u jdbc:hive2://$HS2:10000/default -n $username -p $passwd -f $filename |
nsabharwal
/ ambari auto sync
Created
November 5, 2015 23:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| yum install expect* | |
| #!/usr/bin/expect | |
| spawn ambari-server sync-ldap --existing | |
| expect "Enter Ambari Admin login:" | |
| send "admin\r" | |
| expect "Enter Ambari Admin password:" | |
| send "admin\r" | |
| expect eof |
nsabharwal
/ Phoenix Blog
Created
October 28, 2015 00:32
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| drop table if exists crime; | |
| create table crime ( | |
| caseid varchar, | |
| Date varchar, | |
| block varchar, | |
| description varchar, | |
| sdesc varchar, | |
| ldesc varchar, | |
| arrest char(2), | |
| domestic char(2), |