AK kumar-de

In `~/.aws/credentials` file, add a default profile

[default]
aws_access_key_id = XXXXXXXXXXX
aws_secret_access_key = XXXXXXXXXXXXXXXXXXXXXXXXXXXX

In `~/.aws/config` file, add the following

[default]

import org.apache.spark.sql.execution.datasources.hbase.HBaseTableCatalog
import sqlContext.implicits._

val configuration = s"""{"hbase.zookeeper.quorum":"127.0.0.1","hbase.zookeeper.property.clientPort":"2181"}"""

val tableCatalog = s"""{
                 |"table":{"namespace":"$namespace", "name":"$tableName"},
                 |"rowkey":"key",
 |"columns":{

Update the system

sudo yum -y update

Install GUI (GNOME Desktop)

sudo yum -y groupinstall "GNOME Desktop" "Administration Tools"

Run the following to start GNOME at boot-time

sudo ln -sf /lib/systemd/system/runlevel5.target /etc/systemd/system/default.target

Install EPEL repository

List keystore/truststore entries

keytool -list -v -keystore keystore.jks

Use Java `keytool` to convert from JKS to P12

Export from keytool's proprietary format (called "JKS") to standardized format PKCS #12:

keytool -importkeystore -srckeystore keystore.jks -destkeystore keystore.p12 -deststoretype PKCS12 -srcalias <jkskeyalias> -deststorepass <password> -destkeypass <password>

import logging

logging.basicConfig(format='%(asctime)s - %(process)d - %(levelname)s - %(message)s', datefmt='%d-%b-%y %H:%M:%S', level=logging.INFO)

Reduce log level

logging.getLogger("pika").setLevel(logging.WARNING)

Or, disable propagation

set up Redis used between connector and Datahub for local testing

sudo docker run \
--name umati-redis \
-d \
-p 6379:6379 \
-e REDIS_PASSWORD=mypas \
redis sh \
-c 'exec redis-server --requirepass "$REDIS_PASSWORD"'

Install RabbitMQ on Docker (with management console)

docker run -d --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3-management

Install Pika library

pip3 install pika

Without logs

docker run -d --log-driver none --net=host achintyakumar/docker-image:1.0

With log-rotate

docker run --log-opt max-size=50m --name=datahub-with-logrotate --net=host -d achintyakumar/aggregated-server:0.2

	import org.apache.hadoop.hbase.client.ConnectionFactory
	import org.apache.hadoop.hbase.{HBaseConfiguration, TableName}
	import org.apache.log4j.{BasicConfigurator, Logger}

	object HbaseTest {
	def main(args: Array[String]): Unit = {
	BasicConfigurator.configure()
	val logger = Logger.getLogger(getClass.getName)
	logger.info("hey")
	val config = HBaseConfiguration.create

	#!/bin/bash

	# This script will extract the certificate and key from an .ovpn file
	# into their own files, which makes it possible to use them to configure
	# the VPN using Ubuntu's network manager

	# Usage example:
	# >> ovpnconvert username.dev.ovpn

	# You can keep following these instructions here: