pfreixes

Output of this command [1]

$ python set_memmory_usage.py
1. Differnece between list, dict and set containers with 1M of numbers regarding the size and container overhead
Sizeof with dict type: 48M, overhead per item 50b
Sizeof with set type: 32M, overhead per item 33b
Sizeof with list type: 8M, overhead per item 8b
----------

pfreixes

# More info about ANSI escape sequences
# http://ascii-table.com/ansi-escape-sequences.php

import sys
import random
from time import sleep

ROWS = 10

while True:

pfreixes

+-------------------+-------------------------+---------+---------+---------+-------+
|Name               |Parameters               |     Real|     User|      Sys|  Msg/s|
+-------------------+-------------------------+---------+---------+---------+-------+
|Pika_Threads       |{'threads': 2}           |     3.03|     1.24|     0.15|   1650|
|Pika_Threads       |{'threads': 4}           |     1.78|     1.26|     0.19|   2808|
|Pika_Threads       |{'threads': 8}           |     1.48|     1.12|     0.16|   3378|
|Pika_Threads       |{'threads': 16}          |     1.43|     1.10|     0.27|   3496|
|Pika_Threads       |{'threads': 32}          |     1.31|     1.14|     0.30|   3816|
|Pika_Async         |{'connections': 2}       |     2.75|     0.96|     0.07|   1818|
|Pika_Async         |{'connections': 4}       |     1.98|     0.88|     0.09|   2525|

pfreixes

1 publisher > 1 queue
queues = 10K
messages per queue = 100
concurrence consumers = 20
queues binded by consumer = 50
qos by consumer = 40
asyncronous pattern

Throughput got 16K messages, why ?

pfreixes

Size operation arround 50 bytes, write concern is acknowledgment.

Bulk trhoughput vs number of update operations

2	4	8	16	32	64	128	256	512
4.5K	9K	12K	16K	19k	22K	23K	24K	27K

pfreixes

Leave ready the DB with a set of 10K documents with a length arround of 1K

from pymongo import MongoClient
from bson.objectid import ObjectId

client = MongoClient('localhost', 27017)
db = client['test']
collection = db['test']
collection.remove()

pfreixes

All consumers consuming over 1K queues, growing from 1 to 1K messages.

consumer_threaded_pika.py

import pika
import threading
import sys
import time

pfreixes

N = 100000

# First approximation, brute force
def is_prime(number):
    if number == 2:
        return True
    for x in range(2, number/2+1):
        if number % x == 0:
            return False
    return True

pfreixes

#!/usr/bin/env python

UPPER_ITERATIONS = 10000000
    
def fac(n):
    if n == 0:
        return 1 
    return n * fac(n - 1)

pfreixes

Boolean are integers, integers are immutable and immutable objects are shared along the life cycle of your programm. Then it comes true

>>> False is False
True
>>> 123 is 123
True
>>> [1] is [1]
False