Skip to content

Instantly share code, notes, and snippets.

View erwan-lemonnier's full-sized avatar

Erwan erwan-lemonnier

View GitHub Profile
@erwan-lemonnier
erwan-lemonnier / do_add_friend.py
Last active November 1, 2020 06:31
Bulk operation via a smart cache (proof of concept)
from pymacaron.auth import get_userid
# Implements an hypothetical endpoint that let's a user (identified by a JWT token)
# add another user as friend (identified by its user ID in the data object received
# as parameter, mapping to the POST request's body)
def do_add_friend(data):
viewer_id = get_userid() # Extract the ID of the current user from its JWT auth token
friend_id = data.user_id # And 'user_id' in the POST request is the friend's ID
@erwan-lemonnier
erwan-lemonnier / cache_and_saver_pattern.py
Last active November 1, 2020 06:17
Bulk fetch from Datastore only if objects aren't already in the cache
# Here is a more complicated endpoint that retrieve objects
# from datastore in multiple phases, and build up a queue of objects
# to bulk commit at the end
from pymacaron.auth import get_userid
def do_get_recommended_new_friends(data):
cache = ObjectCache()
saver = ObjectSaver(cache=cache)
@erwan-lemonnier
erwan-lemonnier / message.json
Last active November 28, 2020 03:33
A basic message structure in a chat
{
"message_id": "1234567890",
"chat_id": "chat-123",
"author_id": "user-456",
"text": "this is a message",
"date_created": "2020-11-28 13:15:09+00:00"
}
@erwan-lemonnier
erwan-lemonnier / message-advanced.json
Last active November 28, 2020 10:47
a more advanced message structure
{
"message_id": "1234567890",
"chat_id": "chat-123",
"thread_id": "branch-789",
"author_id": "user-456",
"type": "TEXT",
"text": "this is a message",
"image_url": null,
"count_views": 99,
"count_likes": 22,
@erwan-lemonnier
erwan-lemonnier / data-migration.py
Last active November 28, 2020 03:36
data migration
class Message():
def normalize(self):
# If message lacks a 'likes' counter, initialize one
if not hasattr(self, 'count_likes'):
self.count_likes = 0
# Each time you are fetching a message from datastore, do
# something like:
msg = fetch_from_database(message_id)
@erwan-lemonnier
erwan-lemonnier / tracker.json
Created November 28, 2020 03:40
Simple chat tracker
{
"user_id": "user-123",
"chat_id": "chat-123456",
}
{
"user_id": "user-123",
"chat_id": "chat-789012",
}
@erwan-lemonnier
erwan-lemonnier / tracker-advanced.json
Last active November 28, 2020 10:52
more advanced chat tracker
{
"user_id": "user-123",
"chat_id": "chat-456",
"thread_id": "branch-abc",
"date_created": "2020-11-28 13:15:09+00:00",
"date_last_seen": "2020-11-30 08:26:11+00:00",
"is_muted": false,
"is_active": true,
}
@erwan-lemonnier
erwan-lemonnier / message-added-event.json
Created November 28, 2020 04:12
message added event
{
"timestamp": "2020-11-28T04:11:40.530Z",
"eventId": "string",
"eventType": "MESSAGE_ADDED",
"message": {
"messageId": "string",
"chatId": "string",
"parentId": "string",
"isRead": true,
"type": "USER",
renderer = PageRenderer()
pdf_document = load_from_file(pdf)
emails = []
for page_number in range(pdf_document.pages):
page = pdf_document.create_page(page_number)
image = renderer.render_page(page)
# Convert pdf image to PIL image
# Use vision api to extract all text in the image
response = gvc.annotate_image({
'image': {
'source': {
'image_uri': 'gs://%s/%s' % (bucket_name, IMG_NAME),
}
},
'features': [
{
'type_': vision.Feature.Type.DOCUMENT_TEXT_DETECTION,