Skip to content

Instantly share code, notes, and snippets.

View benwtrent's full-sized avatar
🏠
Working from home

Benjamin Trent benwtrent

🏠
Working from home
View GitHub Profile
@benwtrent
benwtrent / recreation_attempt.es
Last active February 26, 2020 17:32
Trying to recreate wierd script bug
PUT programs
{
"mappings": {
"properties": {
"visit_times" : {
"type" : "date",
"null_value" : "0"
}
}
}
@benwtrent
benwtrent / pipeline_and_composite
Created February 11, 2020 14:43
pipeline agg in composite example
GET kibana_sample_data_ecommerce/_search
{
"size": 0,
"aggs": {
"user": {
"composite": {
"sources": [
{
"user": {
"terms": {
@benwtrent
benwtrent / ESModel.py
Last active February 11, 2020 17:53
model transformers
from typing import List
def add_if_exists(d: dict, k: str, v) -> dict:
"""
:param v:
:param k:
:type d: object
"""
if v is not None:
"processors": [
{
{
"drop": {
"if" : "ctx.lat == 0.0"
}
}
},
{
"set": {
@benwtrent
benwtrent / mappings
Last active November 11, 2019 17:24
building out avg price prediction on a house given ashville air bnb listing data
{
"listings-ash" : {
"aliases" : { },
"mappings" : {
"_meta" : {
"created_by" : "ml-file-data-visualizer"
},
"properties" : {
"@timestamp" : {
"type" : "date"
@benwtrent
benwtrent / LogParser.java
Created July 12, 2019 19:37
Java implementation of the Drain algorithm.
import java.util.AbstractMap;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;
/**
* This class contains the parsing logic for implementing the DRAIN algorithm.
@benwtrent
benwtrent / m2cgen_painless.py
Last active May 9, 2019 21:03
This is a giant script created via some hacking against m2cgen
import xgboost as xgb
from sklearn import datasets
from sklearn.metrics import mean_squared_error
import m2cgen as m2c
diabetes = datasets.load_diabetes() # load data
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(diabetes.data, diabetes.target, test_size=0.2, random_state=0)
print(diabetes.feature_names)
@benwtrent
benwtrent / painless_linear_regression.json
Created March 7, 2019 21:50
Linear Regression Model inference with Painless in ElasticSearch
PUT _scripts/linear_regression_inference
{
"script": {
"lang": "painless",
"source": """
double total = params.intercept;
for (int i = 0; i < params.coefs.length; ++i) {
total += params.coefs.get(i) * doc[params['x'+i]].value;
}
return total;
@benwtrent
benwtrent / values_true.json
Created February 28, 2019 19:25
How to select terms buckets by all values being true
PUT test-bool/
{
"mappings" : {
"_doc" : {
"properties" : {
"my-field" : {
"type" : "boolean"
}, "my-terms": {
"type" : "keyword"
}
@benwtrent
benwtrent / mat-to-csv
Created January 3, 2019 19:25
mat to csv octave: just a simple
f=load(FILENAME.mat)
C = [f.X, f.y]
csvwrite(FILENAME.csv, C)