Skip to content

Instantly share code, notes, and snippets.

@Barry1
Last active April 6, 2022 10:33
Show Gist options
  • Save Barry1/9012c7950f32a970f22bf38cc45145b1 to your computer and use it in GitHub Desktop.
Save Barry1/9012c7950f32a970f22bf38cc45145b1 to your computer and use it in GitHub Desktop.
AsynIO, Threads or Processes - what to use when and how

Managing Concurrency in Python

Just to mention before - not seldom words are to be understood carefully. I use the understanding to be found in Definitions.

Quick result first

Any form would have its advantages. A quick decision table could be drafted like

Concurrency Type Features Use Criteria
AsyncIO Single process, single thread, cooperative multitasking, tasks cooperatively decide switching. Slow I/O-bound
Threading Single process, multiple threads, pre-emptive multitasking, OS decides task switching. Fast I/O-bound
Multiprocessing Multiple processes, high CPU utilization. CPU-bound

inspired by leimao.

Definitions

Concurrency : Simultaneous processing for sequences of instructions. Nothing is said on beeing real parallel. Simultaneous could also be done by pre-emptive processing (interrupting one sequence and go further with another).

Parallelism : This is like Concurrency but things have to be done real parallel, thus guaranteed not pre-emptive. You will in any case need more than one CPU for real parallelism.

Task : This is just a word describing something to do, like a job.

Thread : This is a seperated concept for I/O-Bound processing. It enables preemptive multi processing.

Process : This is a separated conecept for CPU-Bound processing

Executor : This is a concept for driving parallelism. You can create an executor and hand over tasks to be scheduled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment