Skip to content

Instantly share code, notes, and snippets.

View engineerball's full-sized avatar
🏠
Working from home

engineerball engineerball

🏠
Working from home
View GitHub Profile
This file has been truncated, but you can view the full file.
127.0.0.1 0000.en-icloud.com
127.0.0.1 0000y.cn
127.0.0.1 001995.signin-apple.com
127.0.0.1 0041-3413.com
127.0.0.1 0041-5413.com
127.0.0.1 004b.cn
127.0.0.1 004j.cn
127.0.0.1 004n.cn
127.0.0.1 0051-4413.com
127.0.0.1 0051-6413.com
@engineerball
engineerball / dataflow-log.txt
Created December 29, 2022 07:09
dataflow-mongodb-cdc-bigquery log
Error message from worker: java.lang.IllegalArgumentException: Unable to encode element 'Document{{_id=Document{{_data=8263A951300000033B2B022C0100296E5A10041F3B43A3680D43F5BE510750C042231346645F6964006463A9512F615203DEA288684B0004}}, operationType=insert, clusterTime=Timestamp{value=7181360347419247419, seconds=1672040752, inc=827}, fullDocument=Document{{_id=63a9512f615203dea288684b, arrRef=[63a9512f615203dea2886508, 63a9512f615203dea2886509, 63a9512f615203dea288650a], ref=63a9512f615203dea288650d}}, ns=Document{{db=mgodatagen_test, coll=link}}, documentKey=Document{{_id=63a9512f615203dea288684b}}}}' with coder 'SerializableCoder(org.bson.Document)'.
org.apache.beam.sdk.coders.Coder.getEncodedElementByteSize(Coder.java:300)
org.apache.beam.sdk.coders.Coder.registerByteSizeObserver(Coder.java:291)
org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.registerByteSizeObserver(WindowedValue.java:642)
org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.registerB
@engineerball
engineerball / install-docker-centos7.sh
Created October 20, 2017 05:42
Docker installation on Centos7
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum-config-manager --enable docker-ce-edge
sudo yum install docker-ce
sudo systemctl enable docker
sudo systemctl start docker
@engineerball
engineerball / worker01-config.yaml
Created February 15, 2017 11:25
Kubernetes on CoreOS
#cloud-config
hostname: worker01
coreos:
etcd2:
discovery: https://discovery.etcd.io/<token-id>
listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
advertise-client-urls: http://172.17.181.248:2379
listen-peer-urls: http://172.17.181.248:2380,http://172.17.181.248:7001
initial-advertise-peer-urls: http://172.17.181.248:2380
data-dir: /var/lib/etcd2
@engineerball
engineerball / master01-config.yaml
Last active February 15, 2017 11:25
Kubernetes on Coreos
#cloud-config
hostname: master01
coreos:
etcd2:
discovery: https://discovery.etcd.io/<token-id>
listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
advertise-client-urls: http://172.17.181.247:2379
listen-peer-urls: http://172.17.181.247:2380,http://172.17.181.247:7001
initial-advertise-peer-urls: http://172.17.181.247:2380
data-dir: /var/lib/etcd2
@engineerball
engineerball / graph-matrix.py
Created November 3, 2016 19:41
extract node detail and matrix from graph xml
from xml.dom import minidom
from signal import signal, SIGPIPE, SIG_DFL
from sys import argv
script, filename = argv
signal(SIGPIPE,SIG_DFL)
edge = []
xmldoc = minidom.parse(filename)
itemlist = xmldoc.getElementsByTagName('node')
print(len(itemlist))
@engineerball
engineerball / logstash.conf
Last active September 28, 2016 09:57
view docker logs in ELK
input {
gelf {}
}
output {
elasticsearch {
hosts => ["elasticsearch"]
workers=> 10
}
stdout {
}
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates git
sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
echo "deb https://apt.dockerproject.org/repo ubuntu-trusty main" | sudo tee /etc/apt/sources.list.d/docker.list
sudo apt-get update
sudo apt-get install -y docker-engine
#sudo systemctl start docker.service
sudo service docker restart
sudo usermod -aG docker $USER
@engineerball
engineerball / Vagrantfile
Last active September 10, 2016 05:44
Vagrant file for devops workshop
Vagrant.configure("2") do |config|
config.vm.define "node-1" do |node|
node.vm.box = "ubuntu/trusty64"
node.vm.hostname = "node-1"
node.vm.network :private_network, ip: "192.168.56.101"
node.vm.provider :virtualbox do |v|
v.customize ["modifyvm", :id, "--memory", 1024]
v.customize ["modifyvm", :id, "--name", "node-1"]
mkdir tmp
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tmp/nginx.key -out tmp/nginx.crt -subj "/CN=nginxsvc/O=nginxsvc"
echo "apiVersion: v1
kind: Secret
metadata:
name: nginxsecret
type: Opaque
data:
nginx.crt: `base64 -w 0 tmp/nginx.crt`