hivefans / post-receive.sh

Last active March 17, 2020 02:03 — forked from icyleaf/post-receive.sh

git autodeploy script when it matches the string "[deploy]"|-|{"files":{"post-receive.sh":{"env":"plain"}},"tag":"bigdata"}

	#!/bin/sh
	#
	# git autodeploy script when it matches the string "[deploy]"
	#
	# @author icyleaf <icyleaf.cn@gmail.com>
	# @link http://icyleaf.com
	# @version 0.1
	#
	# Usage:
	# 1. put this into the post-receive hook file itself below

hivefans / EsPerformanceTest.java

Last active March 17, 2020 02:03

Elasticsearch 批量index性能测试|-|{"files":{"build.gradle":{"env":"plain"},"EsPerformanceTest.java":{"env":"plain"}},"tag":"Uncategorized"}

	package kingsoft.com;

	import org.apache.log4j.BasicConfigurator;
	import org.elasticsearch.action.admin.indices.flush.FlushRequest;
	import org.elasticsearch.action.bulk.BackoffPolicy;
	import org.elasticsearch.action.bulk.BulkProcessor;
	import org.elasticsearch.action.bulk.BulkRequest;
	import org.elasticsearch.action.bulk.BulkResponse;
	import org.elasticsearch.action.index.IndexRequest;
	import org.elasticsearch.client.transport.TransportClient;

hivefans / README.markdown

Last active March 17, 2020 02:03 — forked from karmi/.gitignore

Example Nginx configurations for Elasticsearch|-|{"files":{"passwords":{"env":"plain"},"nginx_load_balancer.conf":{"env":"plain"},"nginx_keep_alive.conf":{"env":"plain"},"admins":{"env":"plain"},"nginx_basic_proxy.conf":{"env":"plain"},"nginx_http_auth_deny_path.conf":{"env":"plain"},"nginx_http_auth_roles.conf":{"env":"plain"},"nginx_http_auth_…

Example Nginx Configurations for Elasticsearch

This repository contains couple of example configurations for using Nginx as a proxy for Elasticsearch.

These examples can be run standalone from this repository -- the general pattern is:

$ nginx -p $PWD/nginx/ -c $PWD/<CONFIG FILE>

When you change the configuration, simply reload the Nginx process to pick up the changes:

hivefans / elasticsearch-for-write-heavy-cluster.yml

Last active March 17, 2020 02:03 — forked from shihpeng/elasticsearch-for-write-heavy-cluster.yml

Elasticsearch configs|-|{"files":{"elasticsearch-for-write-heavy-cluster.yml":{"env":"plain"}},"tag":"bigdata"}

	#
	# Cluster and Node
	# Note: We should always set the cluster name and nodes' name specifically.
	#
	cluster.name: elasticsearch_prod
	node.name: elasticsearch_001
	#node.(attribute): (attibute_value)

	#
	# Index

hivefans / ImproveIndexingPerformance.sh

Last active September 21, 2020 05:35

Increase Indexing Performance - Elasticsearch|-|{"files":{"ImproveIndexingPerformance.sh":{"env":"plain"}},"tag":"bigdata"}

	#!/bin/bash

	#Close index
	curl -XPOST "localhost:9200/$1/_close"


	#refresh rate
	curl -XPUT "localhost:9200/$1/_settings" -d '{
	"index" : {
	"refresh_interval" :-1

hivefans / logstash.index.json

Last active March 17, 2020 02:03 — forked from WPsites/logstash.index.json

Elasticsearch index template for logstash that contains additional NGINX fields|-|{"files":{"logstash.index.json":{"env":"plain"}},"tag":"bigdata"}

	{
	"template_logstash":{
	"template" : "logstash*",
	"settings" : {
	"number_of_shards" : 5,
	"index.cache.field.type" : "soft",
	"index.refresh_interval" : "5s",
	"index.store.compress.stored" : true,
	"index.query.default_field" : "message",
	"index.routing.allocation.total_shards_per_node" : 5

hivefans / flume-ng-agent.sh

Last active March 17, 2020 02:03 — forked from ashrithr/flume-ng-agent.sh

Custom Flume NG Agent INIT script for centos for runnig multiple agents on same machine|-|{"files":{"flume-ng-agent.sh":{"env":"plain"},"usage.md":{"env":"plain"}},"tag":"Uncategorized"}

	#!/bin/bash
	#
	# Licensed to the Apache Software Foundation (ASF) under one or more
	# contributor license agreements. See the NOTICE file distributed with
	# this work for additional information regarding copyright ownership.
	# The ASF licenses this file to You under the Apache License, Version 2.0
	# (the "License"); you may not use this file except in compliance with
	# the License. You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0

hivefans / HBaseNewAPI.scala

Last active March 17, 2020 02:03 — forked from wuchong/HBaseNewAPI.scala

Spark 下操作 HBase 1.0.0 新版API|-|{"files":{"SparkOnHBase.scala":{"env":"plain"},"build.sbt":{"env":"plain"},"HBaseNewAPI.scala":{"env":"plain"}},"tag":"Uncategorized"}

	import org.apache.hadoop.hbase.util.Bytes
	import org.apache.hadoop.hbase.{HColumnDescriptor, HTableDescriptor, TableName, HBaseConfiguration}
	import org.apache.hadoop.hbase.client._
	import org.apache.spark.SparkContext
	import scala.collection.JavaConversions._

	/**
	* HBase 1.0.0 新版API, CRUD 的基本操作代码示例
	**/
	object HBaseNewAPI {

hivefans / flume.conf

Last active March 17, 2020 02:03 — forked from ottomata/flume.conf

|-|{"files":{"flume.conf":{"env":"plain"}},"tag":"bigdata"}

	webrequest.channels = file-channel
	webrequest.sources = udp2log
	webrequest.sinks = hdfs-sink

	# Channel which buffers events on disk
	webrequest.channels.file-channel.type = file
	webrequest.channels.file-channel.checkpointDir = /var/lib/hadoop/data/e/flume/file-channel/checkpoint
	webrequest.channels.file-channel.dataDirs = /var/lib/hadoop/data/e/flume/file-channel/data
	webrequest.channels.file-channel.checkpointInterval = 1000

hivefans / JTarUtils.java

Last active March 17, 2020 02:03 — forked from johnkil/JTarUtils.java

Implementation of two versions of the utilities to decompress tar.gz archives (apache tar & jtar).|-|{"files":{"JTarUtils.java":{"env":"plain"},"TarUtils.java":{"env":"plain"}},"tag":"Uncategorized"}

	import java.io.BufferedInputStream;
	import java.io.BufferedOutputStream;
	import java.io.File;
	import java.io.FileInputStream;
	import java.io.FileOutputStream;
	import java.io.IOException;
	import java.util.zip.GZIPInputStream;

	import org.xeustechnologies.jtar.TarEntry;
	import org.xeustechnologies.jtar.TarInputStream;

hivefans hivefans

Example Nginx Configurations for Elasticsearch