Easiest HDFS cluster in the world with kubernetes.
Inspiration from kimoonkim/kubernetes-HDFS
kubectl create -f namenode.yaml
kubectl create -f datanode.yaml
Setup a port-forward to so you can see it is alive:
import akka.actor.ActorSystem | |
import akka.stream._ | |
import akka.stream.scaladsl._ | |
import scala.io.StdIn | |
import scala.util.Random | |
object SimplePartitionSample extends App { | |
implicit val system = ActorSystem() |
#!/bin/sh | |
sed -i.default "s/^zend_extension=/;zend_extension=/" /usr/local/etc/php/7.1/conf.d/ext-xdebug.ini | |
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.php71.plist | |
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.php71.plist | |
sudo brew services restart php71 | |
echo "xdebug disabled" |
package we.sg.data | |
import cats.kernel.Monoid | |
import eu.timepit.refined.{W, refineMV, refineV} | |
import eu.timepit.refined.api.Refined | |
import eu.timepit.refined.boolean.Not | |
import eu.timepit.refined.string.MatchesRegex | |
object RefinementTypes { |
// | |
// While testing the Elasticsearch 6.x http client works for me, using circe for json, | |
// I added a silly `Entity` type class which describes how to get an ID from an entity. | |
// | |
import java.util.UUID | |
import org.apache.http.HttpHost | |
import org.elasticsearch.action.index.IndexRequest |
#!/bin/sh | |
cd "$(dirname "$0")" | |
touch .git/hooks/pre-commit | |
rm .git/hooks/pre-commit | |
ln -s ../../pre-commit-hook.sh .git/hooks/pre-commit |
module Main where | |
-------------------------------------------------------------------------------- | |
-- | This is the `Maybe` data type: | |
-- | | |
-- | > data Maybe a | |
-- | > = Nothing | |
-- | > | Just a | |
-- | | |
-- | `Maybe` is a sum type that can be parameterized over a given `a` type, such |
Easiest HDFS cluster in the world with kubernetes.
Inspiration from kimoonkim/kubernetes-HDFS
kubectl create -f namenode.yaml
kubectl create -f datanode.yaml
Setup a port-forward to so you can see it is alive:
{ | |
timers: { | |
running: 0, | |
pending: 4 | |
}, | |
configuration: { | |
admin_error_log: "logs/error.log", | |
cassandra_lb_policy: "RoundRobin", | |
admin_access_log: "logs/admin_access.log", | |
cassandra_port: 9042, |
This is a little trick I use to spin up the packages instalation on Debian/Ubuntu boxes in Vagrant.
I add a simple function that checks if a directory named something similar to ~/.vagrant.d/cache/apt/opscode-ubuntu-12.04/partial
(it may have another path in Windows or MacOS) and create the directory if it doesn't already exist.
def local_cache(basebox_name)
cache_dir = Vagrant::Environment.new.home_path.join('cache', 'apt', basebox_name)
partial_dir = cache_dir.join('partial')
partial_dir.mkdir unless partial_dir.exist?
cache_dir
# ======================== Elasticsearch Configuration ========================= | |
# | |
# NOTE: Elasticsearch comes with reasonable defaults for most settings. | |
# Before you set out to tweak and tune the configuration, make sure you | |
# understand what are you trying to accomplish and the consequences. | |
# | |
# The primary way of configuring a node is via this file. This template lists | |
# the most important settings you may want to configure for a production cluster. | |
# | |
# Please see the documentation for further information on configuration options: |