There's so many way to send logs to an elk... logspout, filebeat, journalbeat, etc.
But docker has a gelf log driver and logstash a gelf input. So here we are.
Here is a docker-compose to test a full elk with a container sending logs via gelf.
There's so many way to send logs to an elk... logspout, filebeat, journalbeat, etc.
But docker has a gelf log driver and logstash a gelf input. So here we are.
Here is a docker-compose to test a full elk with a container sending logs via gelf.
#!/bin/bash | |
if [ $# -lt 2 ]; then | |
echo "usage:$0 dev output_dir [iodepth]" | |
echo "example 1: Testing the whole block device. Attention: That will destory the filesystem on the target block device" | |
echo "./run_fio.sh /dev/sdb fio_test" | |
echo "" | |
echo "example 2: Testing a file, but not destory filesystem. Suppose the target device mount on /data" | |
echo "fallocate -l 1G /data/test.dat" | |
echo "./run_fio.sh /data/test.dat fio_test" |
input { | |
file { | |
path => "d:/temp/2020052109.tzt" | |
start_position => "beginning" | |
sincedb_path => "NUL" | |
codec => multiline { | |
pattern => "^%{TIME}" | |
negate => true | |
what => previous |
用法:安装anaconda | |
python szfdc.py > hr4.txt | |
一次一栋,抓其他的要自己改改 urlz 链接 |
from urllib.request import urlopen | |
from bs4 import BeautifulSoup | |
import re | |
count = 1 | |
url0 = "http://zjj.sz.gov.cn/ris/bol/szfdc/projectdetail.aspx?id=49916" | |
#url0 = "http://zjj.sz.gov.cn/ris/bol/szfdc/projectdetail.aspx?id=49818" | |
#url0 = "http://zjj.sz.gov.cn/ris/bol/szfdc/projectdetail.aspx?id=49633" | |
#url0 = "http://zjj.sz.gov.cn/ris/bol/szfdc/projectdetail.aspx?id=49574" | |
html0 = urlopen(url0) |
Anaconda is an excellent, simple way to get Python up and running on your computer. But, it includes a lot of packages you'll never use but consume gigs and gigs of hard drive space. Instead, you can just install miniconda and then choose the individual packages you need. The steps below explain how to do this to set up a Python environment for geospatial data science. These steps are Windows-specific, but the same process works on Mac or Linux (just don't download the wheels from Gohlke - conda/pip install them directly). If you're having trouble, here are more detailed instructions on getting geopandas and geospatial Python up and running.
C:\Anaconda
and set it as the system's default Pythoninput { | |
generator { | |
message => "2020-02-10T15:14:15.003Z" | |
count => 1 | |
} | |
} | |
filter { | |
ruby { |
# this is an alternate compact function implementation which also removes keys with empty string values | |
# writing the tests for this is left as an excercise to the reader :D | |
def compact(h) | |
h.inject({}) do |result, (k, v)| | |
if v.is_a?(Hash) | |
result[k] = compact(v) | |
elsif v.is_a?(String) | |
result[k] = v unless v.empty? | |
elsif !v.nil? |
Latency Comparison Numbers (~2012) | |
---------------------------------- | |
L1 cache reference 0.5 ns | |
Branch mispredict 5 ns | |
L2 cache reference 7 ns 14x L1 cache | |
Mutex lock/unlock 25 ns | |
Main memory reference 100 ns 20x L2 cache, 200x L1 cache | |
Compress 1K bytes with Zippy 3,000 ns 3 us | |
Send 1K bytes over 1 Gbps network 10,000 ns 10 us | |
Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD |
input { | |
beats { | |
port => 5301 | |
} | |
} | |
filter { | |
if [fields][type] == "monthly-indexed.r2web.bsd.alljob" { | |
mutate { | |
gsub => ["message","/"," "] |