THEORY: Example of a simple Single Thread Pool implementation in Java

HowToUseIt.java

public class HowToUseIt {
 
  /**
   * Usually we'll have a single instance per client
   */
  private static final SingleThreadPool THREAD_POOL = new SingleThreadPool();
  
  public void executeAsync() {
    try {
      
      THREAD_POOL
        .runInBackground(new PrintWorker());
      
    } catch (RejectedExecutionException e) {
      throw new RuntimeException("There's a thread running already!");
    }
  }
  
  static class PrintWorker implements Runnable {
    
    @Override
    public void run() {
      System.out.println("executing in background...");
    }
  }
}

NamedThreadFactory.java

package br.com.rponte.util.threads;

import java.util.concurrent.ThreadFactory;
import java.util.concurrent.atomic.AtomicInteger;

/**
 * ThreadFactory que permite definir o nome das threads criadas,
 * facilitando assim tracing e logging das threads.
 *
 * <p>
 * A factory cria threads com nomes na forma de <i>pool-{poolName}-thread-{N}</i>,
 * onde <i>{poolName}</i> é a string fornecida no construtor, e <i>{N}</i> é
 * o número sequencial da thread criada por essa factory.
 *
 * Inspirada na classe <code>Executors.DefaultThreadFactory</code> do JDK6.
 *
 * <p>
 * http://dubravsky.com/b/seven-rules-of-executorservice-usage-in-java-rule-3-name-the-threads-of-a-thread-pool
 * https://stackoverflow.com/questions/6113746/naming-threads-and-thread-pools-of-executorservice
 */
public class NamedThreadFactory implements ThreadFactory {

    private final String poolName;
    private final AtomicInteger threadNumber = new AtomicInteger(1);
   
    public NamedThreadFactory(String poolName) {
        this.poolName = poolName;
    }

    @Override
    public Thread newThread(Runnable r) {
        String threadName = "pool-{poolName}-thread-{N}"
                                .replace("{poolName}", poolName)
                                .replace("{N}", String.valueOf(threadNumber.getAndIncrement()))
                        ;
        Thread t = new Thread(r, threadName);
        t.setDaemon(false);
        t.setPriority(Thread.NORM_PRIORITY);
        return t;
    }
}

SingleThreadPool.java

package br.com.rponte.util.threads;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.SynchronousQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

/**
 * Pool de thread de tamanho máximo 1. Rejeita execução caso já
 * exista uma thread em andamento.
 */
public class SingleThreadPool {
    
    /**
     * Pool de thread de tamanho 1.<br/>
     * A idéia é não permitir mais do que uma execução simultanea de uma thread
     */
    private final ExecutorService THREAD_POOL = new ThreadPoolExecutor(1, 1,
                                                          0L, TimeUnit.MILLISECONDS,
                                                          new SynchronousQueue<Runnable>(), // rejeita tarefas excendentes
                                                          new NamedThreadFactory("my_pool")); // define nome para as threads
    
   
    /**
     * Executa worker em background porém não permite mais do que
     * um worker por vez.
     * 
     * Caso exista um processo em execução uma exceção <code>RejectedExecutionException</code> 
     * será lançada.
     */
    public void runInBackground(Runnable worker) throws RejectedExecutionException {
        THREAD_POOL.submit(worker);
    }
    
}

Becareful when working with Spring Boot's @Async because its default thread pool works with an unbounded queue containing only 8 active threads in the pool (in this case, maxSize is ignored). It means that if the creation of new tasks is way faster than the execution of those tasks your application may suffer an OutOfMemoryError.

It's important to configure the thread pool properly for your workload, something like this:

@EnableAsync
@Configuration
class AsyncConfig {

    @Bean
    public Executor taskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(4);
        executor.setMaxPoolSize(24);
        executor.setQueueCapacity(20);
        executor.setKeepAliveSeconds(60);
        executor.setThreadNamePrefix("async-task-");
        executor.setRejectedExecutionHandler(new ThreadPoolExecutor.AbortPolicy()); // if queue is full, throw a RejectedExecutionException
        return executor;
    }
 }

Or you can configure it through application.properties:

spring.task.execution.pool.core-size=4
spring.task.execution.pool.max-size=24
spring.task.execution.pool.queue-capacity=20
spring.task.execution.pool.keep-alive=60s
spring.task.execution.pool.allow-core-thread-timeout=true
spring.task.execution.shutdown.await-termination=false
spring.task.execution.shutdown.await-termination-period=
spring.task.execution.thread-name-prefix=async-task-

In this article you can see a more detailed example of how to configure the Spring Boot's thread pool.

Another interesting point here is: the thread pool's default configuration (unbounded queue, poolSize=8 etc) is not configured by Spring itself. In fact, it's overwritten by Spring Boot auto-configure feature in its task module. You can see its new defaults in the TaskExecutionProperties class. Also, Spring Boot's documentation talks a little bit about its defaults.

By the way, it's important to know how the Spring's thread pool executor works. I mean, you don't need to trust me or anything I told you above, but here's what the Spring's documentation says about its thread pool executor:

Article: On Parallelism and Concurrency
Thread on Twitter by Nicolai Parlog on Parallelism vs Concurrency

Working with Ruby Threads: How Many Threads Are Too Many?

I showed some heuristics for code that is fully IO-bound or fully CPU-bound. In reality, your application is probably not so clear cut. Your app may be IO-bound in some places, and CPU-bound in other places. Your app may not be bound by CPU or IO. It may be memory-bound, or simply not maximizing resources in any way.

[...]

So, as I said at the beginning of this chapter, the only way to a surefire answer is to measure. Run your code with different thread counts, measure the results, and then decide. Without measuring, you may never find the ‘right’ answer.

• How to Determine Java Thread Pool Size: A Comprehensive Guide | by The Java Trail

Can We Follow a Concrete Formula?

The formula for determining thread pool size can be written as follows:

Number of threads = Number of Available Cores * Target CPU utilization * (1 + Wait time / Service time)

Number of Available Cores: This is the number of CPU cores available to your application. It is important to note that this is not > the same as the number of CPUs, as each CPU may have multiple cores.

Target CPU utilization: This is the percentage of CPU time that you want your application to use. If you set the target CPU utilization too high, your application may become unresponsive. If you set it too low, your application will not be able to fully utilize the available CPU resources.

Wait time: This is the amount of time that threads spend waiting for I/O operations to complete. This can include waiting for network responses, database queries, or file operations.

Service time: This is the amount of time that threads spend performing computation.

Blocking coefficient: This is the ratio of wait time to service time. It is a measure of how much time threads spend waiting for I/O operations to complete relative to the amount of time they spend performing computation.

Example Usage

Suppose you have a server with 4 CPU cores and you want your application to use 50% of the available CPU resources.

Your application has two classes of tasks: I/O-intensive tasks and CPU-intensive tasks.

The I/O-intensive tasks have a blocking coefficient of 0.5, meaning that they spend 50% of their time waiting for I/O operations to complete.

Number of threads = 4 cores * 0.5 * (1 + 0.5) = 3 threads

The CPU-intensive tasks have a blocking coefficient of 0.1, meaning that they spend 10% of their time waiting for I/O operations to complete.

Number of threads = 4 cores * 0.5 * (1 + 0.1) = 2.2 threads

In this example, you would create two thread pools, one for the I/O-intensive tasks and one for the CPU-intensive tasks. The I/O-intensive thread pool would have 3 threads and the CPU-intensive thread pool would have 2 threads.

Benchmark JDBC connectors and Java 21 virtual threads