Demonstrates an example of a work queue fanning-out work items to multiple concurrent workers (goroutines).
The work queue source is lines from a log file in this example, but can be anything. The number of workers is variable and can be adjusted from 1 to 100,000's. As of Go 1.5, the runtime uses all visible CPUs (whatever your operating system considers to be a CPU) to schedule concurrent goroutine execution.
In this example, "work" done on each log file line is simulated by the worker sleeping for a randomized # of seconds from 0-5. When running, you can observe that the workers pull "work" items in an unpredictable order (e.g. you never know which worker will get the next item) and they operate in parallel.
This example also includes 2 variables (dateDict
and ipsToInvestigate
) that multiple goroutines share and write to (potentially simultaneously), and thus must be protected by a mutex/semaphore (mu
).