blink1073

Async PyMongo with Greenlet

June 12, 2023

Proposal

Use greenletio to add asyncio support in PyMongo, with minimal changes to existing classes.

Background

The reason that Motor is not "truly asynchronous" is that we have blocking calls in PyMongo that do not enable cooperative multitasking. These include acquiring thread locks and socket i/o. Even if we were to use non-blocking versions of these, the event loop would still be blocked.

The previous proposal required rewriting the entirety of PyMongo to be asyncio-friendly, and gave a small performance hit for synchronous code, especially noticeable for small documents.

blink1073

from __future__ import annotations
from bson import ObjectId, Decimal128, Binary
from pandas.api.extensions import ExtensionDtype, ExtensionArray, register_extension_dtype
from pandas._typing import type_t
import pyarrow as pa
from typing import Union, Any
import numpy as np
import pandas as pd
import numbers

blink1073

"""Compare the dist file created by a migrated package to one created by the original."""
import argparse
import glob
import os
import shutil
import subprocess
import sys
import tarfile
import zipfile

blink1073

Copyright (c) 2022 Steven Silvester
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this
   list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice,

blink1073

{% set name = "pymongoarrow" %}
{% set version = "0.2.0.dev0" %}


package:
  name: {{ name|lower }}
  version: {{ version }}

source:
  url: file:///home/runner/work/mongo-arrow/mongo-arrow/bindings/python/dist/pymongoarrow-{{ version }}.tar.gz

blink1073

from subprocess import getoutput
import os
from datetime import datetime
from github_activity import generate_activity_md
import re
import sys
from ghapi.core import GhApi

name = sys.argv[-1]
os.chdir(name)

blink1073

Phosphor Meeting Minutes - 2019

Link to recorded videos: https://drive.google.com/drive/folders/1tfJ53NHQyequSHbYbSvs3jIFuWbcYZ9E?usp=sharing

Meeting: https://zoom.us/j/650076676

Covered Packages: algorithm collections messaging signaling

blink1073

When submitting an enhancement proposal, individuals will include the following information in their submission.

The problem that this enhancement addresses. If possible include code or anecdotes to describe this problem to readers.

A brief (1-2 sentences) overview of the enhancement you are proposing. If possible include hypothetical code sample to describe how the solution would work to readers.

A detailed explanation covering relevant algorithms, data structures, an API spec, and any other relevant technical information
A list of pros that this implementation has over other potential implementations.

A list of cons that this implementation has.

blink1073

### Keybase proof

I hereby claim:

  * I am blink1073 on github.
  * I am blink1073 (https://keybase.io/blink1073) on keybase.
  * I have a public key ASCDyw9ILX4MYwb0QEz0FoJCY3PcM49VZGetMmdcf3RsOAo

To claim this, I am signing this object:

blink1073