I've been digging in Cortex, just storing this for later ref:
- Locking object allowing for write once, read many operations
- LockedModel - For granting shared/exclusive (R/W) access to a model resource (model name + model version). Also applies global read lock on the models holder
Also dumping a lot of info on async, asyncio, run_in_threadpool, shared objects in FastAPI
- Best method for modifying global shared object safely - no answer :(
- Sync and Async issues
- Concurrency and async / await in FastAPI
- When you declare a path operation function with normal def instead of async def, it is run in an external threadpool that is then awaited, instead of being called directly (as it would block the server)
- Storing object instances in the app context
- Building our own asyncio functions - coroutines
- about threads issue with fastapi
- Best practice when a single handler calls both async and sync functions - run_in_threadpool
- Executing CPU-bound operations - also run_in_threadpool.
- Converting a regular function to an asynchronous function
- Running a thread safe function with FastApi - comes with the contextvars solution
- Using contextvars to run thread unsafe code in async ? - contextvars
- How To Write Asynchronous Code With Contextvars Properly
- get event loop for the fastapi
- Getting Started With Async Features in Python
- AsyncIO in Context
- Speed Up Your Python Program With Concurrency
- Asyncio Lock or Python Lock Object