SK skolomiiets

Optimizing Django and Celery for Handling Many Concurrent Requests

Handling a high volume of concurrent requests in a Django application with Celery for background tasks can be challenging. This guide will walk you through the necessary steps to optimize your setup for better performance and scalability.

Default Setup with Gunicorn and Celery

By default, Gunicorn with Django and Celery uses synchronous workers to handle web requests and background tasks. This means:

Gunicorn: Uses sync workers which can handle one request at a time per worker.
Celery: Processes tasks synchronously within each worker.

githubimport

Allows you to import Python modules from the top level of a GitHub repository. Basically, golang's import semantics but in Python fashion.

>>> import githubimport
>>> from MineRobber9000.test_modules import blah
>>> blah.foo()
"bar"

An Ansible summary

Jon Warbrick, July 2014, V3.2 (for Ansible 1.7)

Configuration file

intro_configuration.html

First one found from of

	# The goal: create a list of maps of subnet mappings so we don't have to statically hard-code them in aws_lb
	# https://www.terraform.io/docs/providers/aws/r/lb.html#subnet_mapping
	locals {
	# These represent dynamic data we fetch from somewhere, such as subnet IDs and EIPs from a VPC module
	subnet_ids = ["subnet-1", "subnet-2", "subnet-3"]
	eips = ["eip-1", "eip-2", "eip-3"]
	}

	# Here's the hack! The null_resource has a map called triggers that we can set to arbitrary values.
	# We can also use count to create a list of null_resources. By accessing the triggers map inside of

	###################################################
	##
	## Alertmanager YAML configuration for routing.
	##
	## Will route alerts with a code_owner label to the slack-code-owners receiver
	## configured above, but will continue processing them to send to both a
	## central Slack channel (slack-monitoring) and PagerDuty receivers
	## (pd-warning and pd-critical)
	##

	import io
	import sys

	class IteratorFile(io.TextIOBase):
	""" given an iterator which yields strings,
	return a file like object for reading those strings """

	def __init__(self, it):
	self._it = it
	self._f = io.StringIO()

	#!/usr/bin/env bash

	curl --header 'Authorization: token $TOKEN' \
	--header 'Accept: application/vnd.github.v3.raw' \
	--remote-name \
	--location https://api.github.com/repos/owner/repo/contents/path