通过一些问题引导读者了解、熟悉YARN,主要分为以下几大块:
#1. RM
##调度器 内容:FariScheduler, 调度器性能,SLS,Ganglia, 队列,抢占,调度原理, node label
问题1. 怎么保证用户的最低资源使用量
问题2. 如果集群中有多余的资源,已达到最低资源使用量的用户是否可以使用这些资源,此时如果有别的队列有app请求资源并且该队列没有达到最低资源使用量,如何处理
| #!/bin/bash | |
| # Get IP for specified host using 'http://ping.eu/ping', then write it to hosts. | |
| if [ $# -ne 1 ]; then | |
| echo "Usage: $0 host" | |
| echo " e.g. # $0 baidu.com" | |
| exit 0 | |
| fi |
| #! /bin/sh | |
| tmux attach -t chen || tmux new -s chen |
| #!/bin/bash | |
| set -u | |
| set -e | |
| if [ $# -ne 1 ]; then | |
| echo "Usage: $0 hadoop_home_directory" | |
| echo " e.g. $0 /home/test/hadoop-2.4.1" | |
| exit -1 | |
| fi |
| #!/bin/bash | |
| set -u | |
| set -e | |
| if [ $# -ne 2 ]; then | |
| echo "Usage : $0 redis_src_directory port" | |
| echo " e.g. $0 /home/test/redis-stable 6379" | |
| exit -1 | |
| fi |
| package main | |
| import ( | |
| "flag" | |
| "fmt" | |
| "io/ioutil" | |
| "net/http" | |
| "encoding/json" | |
| "strings" | |
| "container/list" |
通过一些问题引导读者了解、熟悉YARN,主要分为以下几大块:
#1. RM
##调度器 内容:FariScheduler, 调度器性能,SLS,Ganglia, 队列,抢占,调度原理, node label
问题1. 怎么保证用户的最低资源使用量
问题2. 如果集群中有多余的资源,已达到最低资源使用量的用户是否可以使用这些资源,此时如果有别的队列有app请求资源并且该队列没有达到最低资源使用量,如何处理
Contribution to open source by hex108:
| #!/bin/bash | |
| set -u | |
| set -e | |
| run_cmd() { | |
| echo "$ $@" | |
| eval "$@" | |
| if [ $? -ne 0 ] ; then | |
| echo "Failed to run $@, exiting..." |
| #!/bin/bash | |
| set -e | |
| set -u | |
| #set -x | |
| SSH_KEY="05:8b:8f:f5:09:35:ae:61:a9:3b:21:ea:1c:36:bf:0c" | |
| USER_DATA_FILE="$(dirname $0)/vps.userdata" | |
| ### sub functions |
| Vagrant.configure("2") do |config| | |
| config.vm.box = "ubuntu/xenial64" | |
| config.vm.define "node0" do |node0| | |
| node0.vm.provision :shell, inline: "hostname node0" | |
| end | |
| config.vm.define "node1" do |node1| | |
| node1.vm.provision :shell, inline: "hostname node1" | |
| end |