Skip to content

Instantly share code, notes, and snippets.

@tripleee
tripleee / tails.txt
Created March 25, 2019 12:46
Common URL tails in Metasmoke
4187 reviews/
2667 cream/
2199 review/
1921 enhancement/
1643 serum/
1093 diet/
1064 garcinia/
1052 keto/
882 forskolin/
792 xl/
@tripleee
tripleee / pulse-ec2-installs.md
Last active May 18, 2018 11:00
PulseMonitor investigation

Taking some notes to help myself remember what I installed on the AMI to troubleshoot PulseMonitor.

sudo yum install -y tmux   # actually not necessary
sudo yum install -y python36

cd
git clone https://github.com/Manishearth/ChatExchange

cd PulseMonitor

python3 -m venv hlenv

@tripleee
tripleee / tumblr.py
Created April 22, 2018 11:25
Metasmoke Tumblr hits
#!/usr/bin/env python3
import json, fileinput
tumblrs = dict()
for line in fileinput.input():
data = json.loads(line)
for rec in data:
for field in 'title', 'body':
@tripleee
tripleee / st
Created January 16, 2018 12:04
Ghoti's Awk statistics
# Adapted from https://stackoverflow.com/a/9790156/874188
st () {
awk 'NR==1{min=max=$1}
{ ++n; sum += $1; sumsq+=$1*$1; if ($1>max) max=$1; if ($1<min) min=$1 }
END { print "n:", n, "min:", min, "max:", max, "sum:", sum, "avg:", sum/n,
"stdev: ", sqrt(sumsq/n - (sum/n)**2) }' "${@--}"; }
@tripleee
tripleee / utf8err.py
Created January 16, 2018 09:37
Display context around UTF-8 UnicodeDecodeError
#!/usr/bin/env python3
def validate(filehandle, filename='-', context=10):
try:
for linenum, line in enumerate(filehandle, 1):
pass
except UnicodeDecodeError as err:
# Show context around error
before = err.object[
max(0, err.start-context):max(0, err.start-1)]
@tripleee
tripleee / dry-run.py
Created November 23, 2017 04:54
SmokeDetector local dry run
#!/usr/bin/env python3
"""
dry-run.py: run findspam analysis on JSON posts from search.
"""
import json
from sys import argv
"""
@tripleee
tripleee / ns-metasmoke56
Last active November 15, 2017 19:45
Stack Exchange spam URL nameserver check
I extracted the name server information from 1000-odd spam posts I had downloaded from Metasmoke.
These are from roughly two months, where I sporadically grab the last 100 posts about once a day,
but not every day (there have been breaks which were a week or more when I was too busy or just
forgot).
Raw data below this top-80 listing.
85 dns2.namecheaphosting.com.
85 dns1.namecheaphosting.com.
@tripleee
tripleee / phone.py
Created October 30, 2017 13:07
Testing SmokeDetector phone rule
import logging
import json
from findspam import FindSpam
from classes._Post import Post
logging.basicConfig(level=logging.DEBUG)
poststring = r'''{"user_link": "https://travel.stackexchange.com/users/69704/dan-jerry",
"is_naa": false, "autoflagged": true, "body": "<p>Welcome to brotherhood Illuminati
where you can become\nrich famous and popular and your life story we be change\ntotally
@tripleee
tripleee / rails db:seed failure
Created October 16, 2017 12:49
Metasmoke rails db:seed output
# rails db:seed
/usr/src/app/config/initializers/appconfig.rb:3: warning: already initialized constant AppConfig
/usr/src/app/config/environments/development.rb:21: warning: previous definition of AppConfig was here
rails aborted!
Redis::CannotConnectError: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
/usr/local/bundle/gems/redis-3.3.3/lib/redis/client.rb:345:in `rescue in establish_connection'
/usr/local/bundle/gems/redis-3.3.3/lib/redis/client.rb:331:in `establish_connection'
/usr/local/bundle/gems/redis-3.3.3/lib/redis/client.rb:101:in `block in connect'
/usr/local/bundle/gems/redis-3.3.3/lib/redis/client.rb:293:in `with_reconnect'
/usr/local/bundle/gems/redis-3.3.3/lib/redis/client.rb:100:in `connect'
@tripleee
tripleee / rails db:schema:load output
Created October 16, 2017 12:03
Metasmoke rails db:schema:load output
# mysql
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)
# ./startdb
[ ok ] Starting MySQL database server: mysqld ..
[info] Checking for tables which need an upgrade, are corrupt or were
not closed cleanly..
# mysql
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)
# rails db:create; echo $?
Database 'metasmoke' already exists