Skip to content

Instantly share code, notes, and snippets.

@realfirst
realfirst / gist:e42fc55060194ac95903
Created September 30, 2014 13:22
jackson nested json parse
JsonNode node = mapper.readValue("[{\"listed_count\":1720,\"status\":{\"retweet_count\":78}}]]", JsonNode.class);
System.out.println(node.findValues("retweet_count").get(0).asInt());
spark-submit --class com.oreilly.learningsparkexamples.scala.BasicParseJsonWithJackson target/scala-2.10/learning-spark-examples-assembly-0.0.1.jar spark://X61:7077 files/pandainfo.json /tmp/scala-json-out
@realfirst
realfirst / ssh.md
Last active December 22, 2015 07:18

SSH-Agent

sudo aura -A envoy-git sudo systemctl enable [email protected]

envoy -t ssh-agent -a ssh_key

eval "$(envoy -p)"

my arch system has installed nearly four years.
because systemd make a lot of change and i find my mpd and ssh-agent can not work as use service
when i run
```
systemctl --user
```
return bus file not exist
@realfirst
realfirst / rpm commands.md
Last active December 25, 2015 02:56
rpm commands
  1. check rpm file info rpm -qip /tmp/zookeeper-3.4.5-2.x86_64.rpm

  2. building rpm srpm http://www.owlriver.com/tips/non-root/

  3. rhel 7 close firewall On Redhat 7 Linux system the firewall run as firewalld daemon

systemctrl stop firewalld

// ch5
val rawData = sc.textFile("file:///search/dje/spark-ml/train_noheader.tsv")
val records = rawData.map(line => line.split("\t"))
records.first()
import org.apache.spark.mllib.regression.LabeledPoint
import org.apache.spark.mllib.linalg.Vectors
val data = records.map { r =>
val trimmed = r.map(_.replaceAll("\"",""))
val label = trimmed(r.size - 1).toInt
  1. use python yum module to get os $releasever and $basearch for yum repository python -c 'import yum, pprint; yb = yum.YumBase(); pprint.pprint(yb.conf.yumvar, width=1)'

  2. only sysnc package of newest from remote repository reposync --newest-only -r repo-id

  3. change linux os default default reoute sudo ip route replace default via 10.0.2.2 dev eth0

hive -hiveconf hive.root.logger=DEBUG,console

@realfirst
realfirst / mysql
Last active December 13, 2017 06:20
SELECT User,Host FROM mysql.user;
DROP USER 'testuser'@'localhost';
* MySql export schema without data
mysqldump -u root -p --no-data dbname > schema.sql
mysqldump -u root -p --no-data azkaban > azkaban-schema.sql
* how-to-import-an-sql-file-using-the-command-line-in-mysql