###RUN SPARK ON 数人云
###单机版
- Mesos运行在host上
- Spark driver 和 executor运行docker上
#####1.搭建Mesos集群环境 详情 ,登录数人云,登录控制台后,通过集群管理创建自己的集群 #####2.运行Spark demo
| # 下载测试包 | |
| git clone https://github.com/brownsys/zookeeper-benchmark.git | |
| # build bench tools | |
| mvn -DZooKeeperVersion=3.4.6 package | |
| # run benchmark | |
| java -cp target/lib/*:target/* edu.brown.cs.zkbenchmark.ZooKeeperBenchmark --conf benchmark.conf |
| #!/bin/sh | |
| ############################################## | |
| data_paths=/srv/dockerdata/debuild:/build # | |
| data_name=debdata # | |
| data_image=user/debdata # | |
| bridge=br0 # | |
| ipfile=/etc/default/docker_ips # | |
| ############################################## |
###RUN SPARK ON 数人云
###单机版
#####1.搭建Mesos集群环境 详情 ,登录数人云,登录控制台后,通过集群管理创建自己的集群 #####2.运行Spark demo
| # ZooKeeper | |
| $ sudo docker run --net=host jplock/zookeeper:3.4.6 | |
| #Mesos-Master | |
| ## It's important to specify --ip=<ip> and --hostname=<ip> parameters | |
| $ sudo docker run --net=host mesosphere/mesos-master:0.21.0-1.0.ubuntu1404 --ip=10.42.0.1 --hostname=10.42.0.1 --zk=zk://10.42.0.1:2181/mesos --work_dir=/opt/ --quorum=1 | |
| #Mesosphere-Marathon |
Drop memory-cg.conf into /etc/init. The upstart job will ensure
that the memory cgroup is configured with hierarchical stats tracking
before systemd-logind and/or docker start creating cgroups.
(Yes, it's mostly a crude hack with an obscure way of syncing with cgroup-lite. That can't be helped, though)
| # /etc/security/limits.conf | |
| * soft nofile 999999 | |
| * hard nofile 999999 | |
| root soft nofile 999999 | |
| root hard nofile 999999 | |
| =========================================================== | |
| # /etc/sysctl.conf | |
| # sysctl for maximum tuning |
| # https://coreos.com/docs/running-coreos/platforms/openstack/ | |
| Step one: | |
| ``` | |
| $ wget http://alpha.release.core-os.net/amd64-usr/current/coreos_production_openstack_image.img.bz2 | |
| $ bunzip2 coreos_production_openstack_image.img.bz2 | |
| $ glance image-create --name CoreOS \ | |
| --container-format bare \ | |
| --disk-format qcow2 \ |
| mkdir panamax | |
| cd panamax | |
| curl -O http://download.panamax.io/installer/pmx-installer-latest.zip | |
| unzip pmx-installer-latest.zip | |
| sudo su | |
| ./coreos install --stable |
| // This will only work in environments that support ES6 Proxies. | |
| // As of right now, that's pretty much only Firefox. | |
| function Refrigerator(foods) { | |
| this.foods = foods; | |
| return new Proxy(this, { | |
| get: function (receiver, name) { | |
| if (name in receiver) { |
| =Navigating= | |
| visit('/projects') | |
| visit(post_comments_path(post)) | |
| =Clicking links and buttons= | |
| click_link('id-of-link') | |
| click_link('Link Text') | |
| click_button('Save') | |
| click('Link Text') # Click either a link or a button | |
| click('Button Value') |