hivefans / ImproveIndexingPerformance.sh

Last active September 21, 2020 05:35

Increase Indexing Performance - Elasticsearch|-|{"files":{"ImproveIndexingPerformance.sh":{"env":"plain"}},"tag":"bigdata"}

	#!/bin/bash

	#Close index
	curl -XPOST "localhost:9200/$1/_close"


	#refresh rate
	curl -XPUT "localhost:9200/$1/_settings" -d '{
	"index" : {
	"refresh_interval" :-1

hivefans / elasticsearch-for-write-heavy-cluster.yml

Last active March 17, 2020 02:03 — forked from shihpeng/elasticsearch-for-write-heavy-cluster.yml

Elasticsearch configs|-|{"files":{"elasticsearch-for-write-heavy-cluster.yml":{"env":"plain"}},"tag":"bigdata"}

	#
	# Cluster and Node
	# Note: We should always set the cluster name and nodes' name specifically.
	#
	cluster.name: elasticsearch_prod
	node.name: elasticsearch_001
	#node.(attribute): (attibute_value)

	#
	# Index

hivefans / squid-elasticsearch.json

Last active March 17, 2020 02:03

squid日志导入elasticsearch的模板设置|-|{"files":{"squid-elasticsearch.json":{"env":"plain"}},"tag":"bigdata"}

	{
	"order": 0,
	"template": "hangout-*",
	"settings": {
	"index": {
	"number_of_shards": "40",
	"codec": "best_compression",
	"refresh_interval": "30s"
	}
	},

hivefans / README.markdown

Last active March 17, 2020 02:03 — forked from karmi/.gitignore

Example Nginx configurations for Elasticsearch|-|{"files":{"passwords":{"env":"plain"},"nginx_load_balancer.conf":{"env":"plain"},"nginx_keep_alive.conf":{"env":"plain"},"admins":{"env":"plain"},"nginx_basic_proxy.conf":{"env":"plain"},"nginx_http_auth_deny_path.conf":{"env":"plain"},"nginx_http_auth_roles.conf":{"env":"plain"},"nginx_http_auth_…

Example Nginx Configurations for Elasticsearch

This repository contains couple of example configurations for using Nginx as a proxy for Elasticsearch.

These examples can be run standalone from this repository -- the general pattern is:

$ nginx -p $PWD/nginx/ -c $PWD/<CONFIG FILE>

When you change the configuration, simply reload the Nginx process to pick up the changes:

hivefans / EsPerformanceTest.java

Last active March 17, 2020 02:03

Elasticsearch 批量index性能测试|-|{"files":{"build.gradle":{"env":"plain"},"EsPerformanceTest.java":{"env":"plain"}},"tag":"Uncategorized"}

	package kingsoft.com;

	import org.apache.log4j.BasicConfigurator;
	import org.elasticsearch.action.admin.indices.flush.FlushRequest;
	import org.elasticsearch.action.bulk.BackoffPolicy;
	import org.elasticsearch.action.bulk.BulkProcessor;
	import org.elasticsearch.action.bulk.BulkRequest;
	import org.elasticsearch.action.bulk.BulkResponse;
	import org.elasticsearch.action.index.IndexRequest;
	import org.elasticsearch.client.transport.TransportClient;

hivefans / post-receive.sh

Last active March 17, 2020 02:03 — forked from icyleaf/post-receive.sh

git autodeploy script when it matches the string "[deploy]"|-|{"files":{"post-receive.sh":{"env":"plain"}},"tag":"bigdata"}

	#!/bin/sh
	#
	# git autodeploy script when it matches the string "[deploy]"
	#
	# @author icyleaf <[email protected]>
	# @link http://icyleaf.com
	# @version 0.1
	#
	# Usage:
	# 1. put this into the post-receive hook file itself below

hivefans / gist:4d1aba8f2e554e95ddc7

Last active March 17, 2020 02:03 — forked from hiroyuki-sato/gist:c3996c03cfb8e554d6c8

Embulk how to |-|{"files":{"gistfile1.md":{"env":"plain"}},"tag":"bigdata"}

移動しました

Embulk

Embulkってなに？ということから、Embulkのプラグインを開発するところまでをまとめたページです。

Embulkとは

hivefans / s.sh

Last active March 17, 2020 02:03 — forked from xurenlu/s.sh

ssh机器管理|-|{"files":{"s.sh":{"env":"plain"}},"tag":"bigdata"}

	#! /bin/bash
	#设置console的颜色
	TPUT=/usr/bin/tput
	RED=`$TPUT setaf 1`
	GREEN=`$TPUT setaf 2`
	YELLOW=`$TPUT setaf 3`
	NORM=`$TPUT op`
	HOSTFILE=~/.sshhosts

	#输出红色文字,错误提示用

hivefans / etchosts.sh

Last active March 17, 2020 02:03 — forked from mikeflynn/etchosts.sh

An /etc/hosts manager bash script (v1.1) -- Added import and export commands!|-|{"files":{"etchosts.sh":{"env":"plain"}},"tag":"bigdata"}

	#!/bin/bash

	HOSTSFILE="/etc/hosts"
	BAKFILE="$HOSTSFILE.bak"
	DOMAINREGEX="^[a-zA-Z0-9]{1}[a-zA-Z0-9\.\-]+$"
	IPREGEX="^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$"
	URLREGEX="^https?:\/\/[a-zA-Z0-9]{1}[a-zA-Z0-9\/\.\-]+$"

	backup()
	{

hivefans / Commands.sh

Last active March 17, 2020 02:03 — forked from ottomata/Commands.sh

kafka failed broker tests|-|{"files":{"Commands.sh":{"env":"plain"},"sequence_check.sh":{"env":"plain"},"Results for 10 producers, 10 brokers, 1 failed broker":{"env":"plain"},"sequence_generate.sh":{"env":"plain"}},"tag":"Uncategorized"}

	# Asynchronous (batched) producer:
	./sequence_generate.sh 20000 10000 \| bin/kafka-console-producer.sh --zookeeper analytics1003:2181 --topic test6

	# Synchronous producer:
	./sequence_generate.sh 20000 10000 \| bin/kafka-producer-shell.sh --props config/producer.properties --topic test6

	# Consumer
	bin/kafka-consumer-shell.sh --props config/consumer.properties --topic test6 \| tee -a /tmp/test6.log

	# sequence check

hivefans hivefans

Example Nginx Configurations for Elasticsearch

Embulk

Embulkとは