Skip to content

Instantly share code, notes, and snippets.

View dmukhg's full-sized avatar

Dipanjan Mukherjee dmukhg

  • Bangalore, India
View GitHub Profile
class Crawler:
def __init__(self, base_url, writer, predicate):
"""
* base_url is the first URL that would be crawled.
* writer is a class that is defined below. It is in-charge of writing
to the file system.
* predicate is a function with signature predicate(url). It
returns true, false and on the basis of that, the decision on
whether to download a link or not is taken. """
@dmukhg
dmukhg / latex-control-char-replace.py
Created August 29, 2016 19:39
latex-control-char-replace.py uses regexes to replace latex control characters in tags other than tex-math
import re
from lxml import etree
def _escape_latex_control_chars_in(text):
text = re.sub('#', r'\#', text)
text = re.sub('{', r'\{', text)
text = re.sub('}', r'\}', text)
text = re.sub('~', r'\\textasciitilde', text)
text = re.sub('_', r'\_', text)
@dmukhg
dmukhg / bucky-server.service
Last active November 7, 2016 11:08
BuckyServer SystemD service file
[Unit]
Description=BuckyServer for client metrics RUM
After=network.target
[Service]
PIDFile=/run/bucky-server.pid
User=bucky
WorkingDirectory=/usr/share/bucky-server
ExecStart=/usr/bin/nodejs /usr/share/bucky-server/start.js
ExecReload=/bin/kill -s HUP $MAINPID
@dmukhg
dmukhg / testable-service-bad.py
Last active December 5, 2021 18:36
Writing services that are testable
from .auth import AccessControlList
acl = AccessControlList()
class BadBookService(object):
def create_book(self, user, book):
if acl.can_create_book(user):
# ... code to create book
else:
# ... raise an exception
@dmukhg
dmukhg / testable-service-good.py
Created January 15, 2018 06:53
Writing services that are testable
from .auth import AccessControlList
class GoodBookService(object):
def __init__(self, acl_service=AccessControlList()):
self.acl = acl_service
def create_book(self, user, book):
if self.acl.can_create_book(user):
# ... code to create book
else:
import boto3
from moto import mock_dynamodb2
# Requires having created a table called "test-abuse" with only
# a partition key called 'Key'
item = {
"Key": "some-irrelevant_key",
"SetTypeAttribute": {"SS": set([])},
}