Skip to content

Instantly share code, notes, and snippets.

View sinmetal's full-sized avatar

sinmetal sinmetal

96 followers · 25 following
View GitHub Profile
@sinmetal
sinmetal / gaereverseproxy.go
Created February 23, 2016 06:26 — forked from pokstad/gaereverseproxy.go
Google App Engine reverse proxy in Golang
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// HTTP reverse proxy handler
package goengine
import (
"io"
@sinmetal
sinmetal / select-from-auditlog-where-billing.sql
Created February 12, 2016 04:58
BigQuery Audit Logから現在のBillingと、NewBillingの値を算出するQuery
SELECT
protoPayload.authenticationInfo.principalEmail,
protoPayload.serviceData.jobCompletedEvent.job.jobName.jobId,
protoPayload.serviceData.jobCompletedEvent.job.jobStatistics.billingTier,
protoPayload.serviceData.jobCompletedEvent.job.jobStatistics.totalBilledBytes,
ROUND(((protoPayload.serviceData.jobCompletedEvent.job.jobStatistics.totalBilledBytes*5)/1000000000000),2) Cost_In_Dollars_Now,
ROUND(((protoPayload.serviceData.jobCompletedEvent.job.jobStatistics.totalBilledBytes*(5*protoPayload.serviceData.jobCompletedEvent.job.jobStatistics.billingTier))/1000000000000),2) Cost_In_Dollars_New,
protoPayload.serviceData.jobCompletedEvent.job.jobConfiguration.query.query,
FROM TABLE_DATE_RANGE(bigquery.cloudaudit_googleapis_com_data_access_, DATE_ADD(CURRENT_TIMESTAMP(), -3, 'DAY'), CURRENT_TIMESTAMP())
WHERE
@sinmetal
sinmetal / SearchStorageMonitorController.java
Created September 3, 2015 07:35
App Engine Search API のStaticsを見るサンプル
package org.sinmetal.controller;
import java.io.*;
import java.util.*;
import org.slim3.controller.*;
import com.google.appengine.api.search.*;
public class SearchStorageMonitorController extends SimpleController {
@sinmetal
sinmetal / main.go
Created August 13, 2015 06:45
EpochTimeで入っているjsonをstructに読み込むためのサンプル
package main
import (
"encoding/json"
"fmt"
"strconv"
"time"
)
type Hoge struct {
@sinmetal
sinmetal / select_syslog_from_cloudlogging.sql
Created August 8, 2015 01:22
Cloud Logging経由で出力したjsonをViewに構築し直すためのクエリ
SELECT
SUBSTR(
JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE__.id'),
2, LENGTH(JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE__.id')) - 2)AS id,
SUBSTR(
JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE__.kind'),
2, LENGTH(JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE__.kind')) - 2)AS kind,
SUBSTR(
JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE__.selfLink'),
2, LENGTH(JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE__.selfLink')) - 2)AS selfLink,
@sinmetal
sinmetal / select_json_from_syslog.sql
Created July 31, 2015 10:59
syslogにjson出力して、bigqueryでview作るためのsql
SELECT
JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE__.id') AS id,
JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE__.kind') AS kind,
JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE__.selfLink') AS selfLink,
JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE__.name') AS name,
JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE__.bucket') AS bucket,
JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE__.generation') AS generation,
JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE__.metageneration') AS metageneration,
JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE__.contentType') AS contentType,
JSON_EXTRACT(REGEXP_REPLACE(textPayload, r'.*{"__SAMPLE__', '{"__SAMPLE__'), '$.__SAMPLE_
@sinmetal
sinmetal / select_cloud_logging.sql
Created July 26, 2015 10:38
SELECT
JSON_EXTRACT(protoPayload.line.logMessage, '$.__DS__KIND__PUGEVENT__.id') AS id,
JSON_EXTRACT(protoPayload.line.logMessage, '$.__DS__KIND__PUGEVENT__.organizationId') AS organizationId,
JSON_EXTRACT(protoPayload.line.logMessage, '$.__DS__KIND__PUGEVENT__.title') AS title,
JSON_EXTRACT(protoPayload.line.logMessage, '$.__DS__KIND__PUGEVENT__.description') AS description,
JSON_EXTRACT(protoPayload.line.logMessage, '$.__DS__KIND__PUGEVENT__.url') AS url,
JSON_EXTRACT(protoPayload.line.logMessage, '$.__DS__KIND__PUGEVENT__.startAt') AS startAt,
JSON_EXTRACT(protoPayload.line.logMessage, '$.__DS__KIND__PUGEVENT__.createdAt') AS createdAt,
JSON_EXTRACT(protoPayload.line.logMessage, '$.__DS__KIND__PUGEVENT__.updatedAt') AS updatedAt,
protoPayload.line.logMessage AS source
@sinmetal
sinmetal / QueryAsyncController.java
Created July 8, 2015 05:10
https://cloud.google.com/appengine/docs/java/datastore/async
package org.sinmetal.controller;
import java.io.*;
import java.util.*;
import java.util.logging.*;
import org.codehaus.jackson.*;
import org.codehaus.jackson.map.*;
import org.sinmetal.meta.*;
import org.sinmetal.model.*;
@sinmetal
sinmetal / zsh_curl_loop.sh
Created July 1, 2015 08:28
#!/usr/local/bin/zsh
while true;
do
curl $1
sleep 1
done
@sinmetal
sinmetal / count_ehomaki.sql
Created February 5, 2015 02:16
Tweetを適当にためたBigQueryのTableから、恵方巻関連のつぶやきを拾ったもの。
SELECT
year, month, day,
count(text) AS count
FROM
(
SELECT
YEAR(DATE_ADD (createdAt, 9, "HOUR")) AS year,
MONTH(DATE_ADD (createdAt, 9, "HOUR")) AS month,
DAY(DATE_ADD (createdAt, 9, "HOUR")) AS day,
text