dvasilen

public class HCatInputFormat extends InputFormat<SerializableWritable<Writable>, HCatRecord> {
	
	private final org.apache.hcatalog.mapreduce.HCatInputFormat input;
	
	public HCatInputFormat() {
		input = new org.apache.hcatalog.mapreduce.HCatInputFormat();
	}

	@Override
	public RecordReader<SerializableWritable<Writable>, HCatRecord> createRecordReader(

dvasilen

Extend Ambari

Introduction

Hortonworks wrote a great article how to add new components into Amabri:

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=38571133

Unfortunately, this document is not enough if you really want to add your own component.

dvasilen

##Reactive System Design Links

#Articles and Papers

• The Reactive Manifesto: http://www.reactivemanifesto.org/
• https://en.wikipedia.org/wiki/Reactive_programming
• https://en.wikipedia.org/wiki/Functional_reactive_programming
• Design Methods for Reactive Systems book slides: http://booksite.elsevier.com/9781558607552/slides/slides.pdf
• Programming without a callstack Event Driven Architectures By G Hohpe: http://www.eaipatterns.com/docs/EDA.pdf
• On Distributed Memory Systems: http://blog.paralleluniverse.co/2012/07/10/on-distributed-memory/
• Disruptor source code, papers and articles: https://lmax-exchange.github.io/disruptor/

dvasilen

1. Stop Ambari server
2. Log on to ambari server host shell
3. Run 'psql -U ambari-server ambari'
4. Enter password ****  (this password is stored in
/etc/ambari-server/conf/password.dat)
5. In psql:
update ambari.users set
user_password='538916f8943ec225d97a9a86a2c6ec0818c1cd400e09e03b660fdaaec4af29ddbb6f2b1033b81b00'
where user_name='admin'
6. Quit psql

dvasilen

// Generate airflow config to stdout.
// To start service the from scratch, put this in src/airflow_config/airflow_config.go and run:
// export GOPATH=$(pwd)
// export PATH=$PATH:$GOPATH/bin
// go get k8s.io/client-go/1.4/kubernetes
// go install airflow_config && airflow_config > airflow.yaml
// kubectl create -f airflow.yaml
package main

import (

dvasilen

package main

import (
	"os"

	"k8s.io/client-go/1.4/pkg/api"
	"k8s.io/client-go/1.4/pkg/runtime/serializer/json"
	"k8s.io/client-go/1.4/pkg/util/intstr"
)

dvasilen

package main

import (
	"fmt"
	"os"

	"k8s.io/client-go/1.4/pkg/api/unversioned"
	"k8s.io/client-go/1.4/pkg/api/v1"
	"k8s.io/client-go/1.4/pkg/apis/extensions/v1beta1"
	"k8s.io/client-go/1.4/pkg/runtime"

dvasilen

Generate private key (.key)

# Key considerations for algorithm "RSA" ≥ 2048-bit
openssl genrsa -out server.key 2048
    
# Key considerations for algorithm "ECDSA" ≥ secp384r1
# List ECDSA the supported curves (openssl ecparam -list_curves)
openssl ecparam -genkey -name secp384r1 -out server.key

dvasilen

package main

import (
	"bufio"
	"crypto/aes"
	"crypto/cipher"
	"crypto/rand"
	"fmt"
	"io"
	"io/ioutil"

dvasilen

package main

import (
	"crypto/tls"
	"crypto/x509"
	"flag"
	"io/ioutil"
	"log"
	"net/http"
)