Skip to content

Instantly share code, notes, and snippets.

@lemon24
lemon24 / search-trigger-update-time.py
Last active March 13, 2020 12:04
reader full text search prototype #2; more detailed, timing triggers; https://github.com/lemon24/reader/issues/122
import timeit
import shutil
import textwrap
import functools
import bs4
# reader 0.18
from reader import make_reader
@lemon24
lemon24 / seach-shadow-table.py
Created March 14, 2020 19:46
reader full text search prototype #3; shadow table, with timing; https://github.com/lemon24/reader/issues/122
import timeit
import shutil
import textwrap
import functools
import bs4
# reader 0.18
from reader import make_reader
>>> from reader import make_reader
>>>
>>> reader = make_reader('db.sqlite')
>>> reader.add_feed('http://www.hellointernet.fm/podcast?format=rss')
>>> reader.update_feeds()
>>>
>>> entries = list(reader.get_entries(feed='http://www.hellointernet.fm/podcast?format=rss'))
>>> [e.title for e in entries]
['H.I. #108: Project Cyclops', 'H.I. #107: One Year of Weird', ...]
>>>
@lemon24
lemon24 / fts5-search-syntax-errors.py
Last active March 20, 2020 12:31
brute force what kind of syntax errors FTS5 raises; for https://github.com/lemon24/reader/issues/122
import string
import random
from reader import make_reader, SearchError
reader = make_reader("db.sqlite")
def queries():
for i in range(256):
@lemon24
lemon24 / entry-word-counts.py
Created March 23, 2020 14:41
entries by summary/content word count order of magnitude; for https://github.com/lemon24/reader/issues/122
>>> import math, collections
>>> from reader import *
>>>
>>> reader = make_reader('db.sqlite')
>>>
>>> def word_count_magnitude(text):
... if not text:
... return 0
... return int(math.log10(len(text.split())))
...
@lemon24
lemon24 / mutagen-mp4-tags.py
Last active December 2, 2025 00:23
using mutagen to update MP4 tags
import shutil
import mutagen
shutil.copy('original.mp4', 'new.mp4')
# mutagen.File knows how to open any file (works with both MP4 and MP4):
#
# https://mutagen.readthedocs.io/en/latest/user/gettingstarted.html
# https://mutagen.readthedocs.io/en/latest/api/base.html#mutagen.File
import collections
import textwrap
import functools
class Query(collections.OrderedDict):
indent_prefix = ' '
default_separators = dict(WHERE='AND', HAVING='AND')
@lemon24
lemon24 / 00-sqlite3-server.md
Last active February 14, 2024 14:28
Python sqlite3 server using multiprocessing.managers

This gist tracks the creation of an almost fully functional sqlite3 server in Python 3, using only the multiprocessing.managers standard library module.

But why?

  • To see if it can be done.
  • To learn about multiprocessing managers.
  • Aside from whole-database locking, SQLite (maybe, see below) supports table-level locking between multiple connections in the same process to the same database. A "SQLite server" makes it possible for users in different processes to use SQLite connections that live in the same process.

@lemon24
lemon24 / run-reader-update-like-cron-would.py
Created July 1, 2020 08:48
run various reader update commands in parallel, like cron would; https://github.com/lemon24/reader/issues/175
"""
Run `reader update` and `reader update --new-only && reader search update``
in parallel, like cron would, but at a "scaled down" timeframe.
We use vcrpy to avoid making network calls.
https://github.com/lemon24/reader/issues/175
"""
import os
import os
import os.path
import time
import sys
import random
import itertools
from datetime import datetime
from subprocess import run
from threading import Thread
import threading