Snippet to show how easy Huskarl makes it for an agent to learn from multiple environment instances simultaneously and also to parallelize those instances over multiple CPU cores.

huskarl-parallel-environments-snippet.py

# We will be running multiple concurrent environment instances
instances = 16

# Create a policy for each instance with a different distribution for epsilon
policy = [hk.policy.Greedy()] + [hk.policy.GaussianEpsGreedy(eps, 0.1) for eps in np.arange(0, 1, 1/(instances-1))]

# Create Advantage Actor-Critic agent
agent = hk.agent.A2C(model, actions=dummy_env.action_space.n, nsteps=2, instances=instances, policy=policy)

# Create simulation, train and then test
sim = hk.Simulation(create_env, agent)
sim.train(max_steps=5000, instances=instances, max_subprocesses=8)
sim.test(max_steps=1000)