Ra Soibam rasoibam

Golang Workers / Job Queue

A running example of the code from:

This gist creates a working example from blog post, and a alternate example using simple worker pool.

TLDR: if you want simple and controlled concurrency use a worker pool.

This configuration is no longer updated

(C-x means ctrl+x, M-x means alt+x)

The default prefix is C-b. If you (or your muscle memory) prefer C-a, you need to add this to ~/.tmux.conf:

	package main

	const MaxLength = 1 << 20

	var (
	addr = flag.String("listen", ":8000", "listen for requests")
	numprocs = flag.Int("p", runtime.NumCPU(), "number of workers to start")
	maxqueue = flag.Int("q", runtime.NumCPU()*2, "largest queue size")

	jobs chan Job

	user web;

	# One worker process per CPU core.
	worker_processes 8;

	# Also set
	# /etc/security/limits.conf
	# web soft nofile 65535
	# web hard nofile 65535
	# /etc/default/nginx

	func EncodeStreamingVideo(streamingFile io.Reader, request ShouldCanceler) (*os.File, error) {
	outputFilename := generateFilename("mp4")

	// Actually start the command.
	cmd := exec.Command("ffmpeg",
	// Read input from stdin.
	"-i", "-",
	// ... environment-specific ffmpeg options ...
	"-y", outputFilename)