avatarNGU

Summary

SpringBoot chose HikariCP as the default database connection pool for versions 2.0+ and 3.0+ due to its superior performance, reliability, and efficiency, which are achieved through optimizations like FastList, ConcurrentBag, and the use of Javassist for compact bytecode.

Abstract

HikariCP has been selected as the default connection pool for SpringBoot in its more recent versions due to its high performance in managing database connections. This performance is attributed to several key optimizations: FastList, which is more efficient than ArrayList by eliminating range checks and optimizing removal operations; ConcurrentBag, a lock-free concurrent collection that efficiently manages connection pools with features like sparse array design, borrow and return strategy, thread affinity, and spin policy; and the use of Javassist to generate faster dynamic proxies with less bytecode. HikariCP's design choices, such as using ThreadLocal for faster fetching of connections and a compact bytecode approach, contribute to its reputation as one of the fastest connection pool systems available in Java. These features make HikariCP particularly well-suited for high-concurrency environments and have led to its adoption in SpringBoot's core database management infrastructure.

Opinions

  • The author believes that HikariCP's custom FastList is a significant performance improvement over the standard ArrayList due to reduced range checks and efficient removals.
  • The design of ConcurrentBag in HikariCP is highly regarded for its ability to maximize performance and minimize lock contention, which is crucial in high-concurrency environments.
  • The use of a zero-capacity SynchronousQueue for swapping connections in HikariCP is seen as a key feature that enhances object transfer speed.
  • The author suggests that the use of ThreadLocal in HikariCP is beneficial for caching local resource references and reducing lock conflicts, which is particularly effective for single-threaded tasks or with external synchronization.
  • The author emphasizes the importance of understanding advanced concurrent programming concepts, such as CAS (Compare-and-Swap), volatile keyword usage, and the behavior of WeakReference, to fully appreciate the performance optimizations in HikariCP.
  • The overall sentiment is that HikariCP's design and optimizations make it an ideal choice for a connection pool where performance is critical, and it is well-tuned for the specific workload of applications like SpringBoot.

Why SpringBoot choose HikariCP as 2.0+ 3.0+’s default database connection pool

what makes HakariCP so Fast, simple, reliable?

In github page have details Analyses and performace test data so here we skip it. we will go a little deep into the source code.

Conclusion:

  1. FastList replaces ArrayList: HikariCP uses a custom FastList instead of ArrayList. The “get” method of the FastList eliminates range checking logic, and its “remove” method starts the scan from the tail rather than the head. This is because the opening and closing order of a connection is often opposite.
  2. ConcurrentBag: Faster concurrent collection implementation from HikariCP, which has superior performance.
  3. Faster fetching of connections: When the same thread fetches a database connection, it retrieves the connection from ThreadLocal, avoiding concurrent operations.
  4. Compact bytecode: HikariCP uses a third-party Java bytecode modifying library, Javassist, to generate delegate implementations for dynamic proxies. Compared to JDK proxies, this method is faster and generates less bytecode.

Overall, HikariCP applies several optimization techniques specifically focusing on performance and efficiency, making it one of the fastest connection pool systems available in Java.

FastList:

public final class FastList<T> implements List<T>, RandomAccess, Serializable
{
   private static final long serialVersionUID = -4598088075242913858L;

   private final Class<?> clazz;
   private T[] elementData;
   private int size;

   /**
    * Construct a FastList with a default size of 32.
    * @param clazz the Class stored in the collection
    */
   @SuppressWarnings("unchecked")
   public FastList(Class<?> clazz)
   {
      this.elementData = (T[]) Array.newInstance(clazz, 32);
      this.clazz = clazz;
   }

   /**
    * Construct a FastList with a specified size.
    * @param clazz the Class stored in the collection
    * @param capacity the initial size of the FastList
    */
   @SuppressWarnings("unchecked")
   public FastList(Class<?> clazz, int capacity)
   {
      this.elementData = (T[]) Array.newInstance(clazz, capacity);
      this.clazz = clazz;
   }

   /**
    * Add an element to the tail of the FastList.
    *
    * @param element the element to add
    */
   @Override
   public boolean add(T element)
   {
      if (size < elementData.length) {
         elementData[size++] = element;
      }
      else {
         // overflow-conscious code
         final var oldCapacity = elementData.length;
         final var newCapacity = oldCapacity << 1;
         @SuppressWarnings("unchecked")
         final var newElementData = (T[]) Array.newInstance(clazz, newCapacity);
         System.arraycopy(elementData, 0, newElementData, 0, oldCapacity);
         newElementData[size++] = element;
         elementData = newElementData;
      }

      return true;
   }

   /**
    * Get the element at the specified index.
    *
    * @param index the index of the element to get
    * @return the element, or ArrayIndexOutOfBounds is thrown if the index is invalid
    */
   @Override
   public T get(int index)
   {
      return elementData[index];
   }

   /**
    * Remove the last element from the list.  No bound check is performed, so if this
    * method is called on an empty list and ArrayIndexOutOfBounds exception will be
    * thrown.
    *
    * @return the last element of the list
    */
   public T removeLast()
   {
      T element = elementData[--size];
      elementData[size] = null;
      return element;
   }

   /**
    * This remove method is most efficient when the element being removed
    * is the last element.  Equality is identity based, not equals() based.
    * Only the first matching element is removed.
    *
    * @param element the element to remove
    */
   @Override
   public boolean remove(Object element)
   {
      for (var index = size - 1; index >= 0; index--) {
         if (element == elementData[index]) {
            final var numMoved = size - index - 1;
            if (numMoved > 0) {
               System.arraycopy(elementData, index + 1, elementData, index, numMoved);
            }
            elementData[--size] = null;
            return true;
         }
      }

      return false;
   }

   /**
    * Clear the FastList.
    */
   @Override
   public void clear()
   {
      for (var i = 0; i < size; i++) {
         elementData[i] = null;
      }

      size = 0;
   }

   /**
    * Get the current number of elements in the FastList.
    *
    * @return the number of current elements
    */
   @Override
   public int size()
   {
      return size;
   }

   /** {@inheritDoc} */
   @Override
   public boolean isEmpty()
   {
      return size == 0;
   }

   /** {@inheritDoc} */
   @Override
   public T set(int index, T element)
   {
      T old = elementData[index];
      elementData[index] = element;
      return old;
   }

   /** {@inheritDoc} */
   @Override
   public T remove(int index)
   {
      if (size == 0) {
         return null;
      }

      final T old = elementData[index];

      final var numMoved = size - index - 1;
      if (numMoved > 0) {
         System.arraycopy(elementData, index + 1, elementData, index, numMoved);
      }

      elementData[--size] = null;

      return old;
   }

   /** {@inheritDoc} */
   @Override
   public boolean contains(Object o)
   {
      throw new UnsupportedOperationException();
   }

   /** {@inheritDoc} */
   @Override
   public Iterator<T> iterator()
   {
      return new Iterator<>() {
         private int index;

         @Override
         public boolean hasNext()
         {
            return index < size;
         }

         @Override
         public T next()
         {
            if (index < size) {
               return elementData[index++];
            }

            throw new NoSuchElementException("No more elements in FastList");
         }
      };
   }

   ...

}

FastList is a custom object collection that was used by HikariCP instead of the standard ArrayList provided by Java. The primary motivation for developing FastList and using it in HikariCP was performance optimization.

Here are the main reasons why FastList is used:

  1. Reduced range checks: The standard ArrayList in Java performs range checks every time an element is fetched, i.e., it verifies if the index passed is within the valid range. However, the FastList.get() method eliminates these range-checking logic, which improves the efficiency of retrievals.
  2. Efficient removals: In the standard ArrayList, when an element is removed, all elements to its right are shifted to fill up the space, which could be costly in terms of time for large lists. However, FastList’s remove method starts scanning from the tail rather than the head. This is particularly efficient for stack-like behavior that connection pools often exhibit, where the last (most recent) connection is the first one to be removed (LIFO — Last In First Out).
  3. Fewer mutations: FastList also optimizes away some of the ArrayList’s internal state mutations that can cause extra memory barriers and disrupt CPU cache coherency.
  4. FastList is non-thread-safe: While this might seem like a disadvantage in multi-threaded contexts, for single-threaded tasks or with external synchronization, this can offer performance benefits.

These improvements make FastList a better choice than ArrayList for specific use-cases like a connection pool where performance is highly critical. Moreover, FastList is fine-tuned and optimized for HikariCP’s specific workload, contributing to its claim of being the “fastest” connection pool.

Please note, the usage of ArrayList or FastList may vary across different versions of HikariCP and can be subject to changes according to their release notes.

ConcurrentBag：

It’s a lock free concurrent collection used to hold and manage the pool of database connections.

  1. SparseArray Design: This class makes use of an internal data structure, an array of IConcurrentBagEntry (connection instances), often referred to as a SparseArray in some discussions. This design allows efficient array traversals and keeps a low memory profile.
  2. Borrow & Return Strategy: Each thread that wants to borrow a connection reserves it first (using a CAS instruction, which stands for Compare-and-Swap and is an atomic instruction). If the reservation is successful (meaning the connection isn’t already reserved by another thread), that connection is borrowed by the thread. This approach significantly reduces the need for locks, enhancing performance.
  3. Thread Affinity Strategy: The ConcurrentBag maintains a certain level of thread affinity. Essentially, this means that connections returned to the pool by a thread are more likely to be served back to the same thread in subsequent requests, thus optimizing the allocation efficiency.
  4. Spin Policy: Another key feature of ConcurrentBag is its spin policy. Instead of putting threads to sleep (expensive operation) when there are no more available connections, it spins (busy-wait) for a brief amount of time. This increases the chances that a connection gets returned, thus improving performance.

The design of ConcurrentBag in HikariCP supports maximizing performance, minimizing lock contention, and reducing thread scheduling overhead, making it an ideal choice for handling high-concurrency environments typical in connection pool scenarios.

public class ConcurrentBag<T extends IConcurrentBagEntry> implements AutoCloseable
{
   private static final Logger LOGGER = LoggerFactory.getLogger(ConcurrentBag.class);


   //Used to cache all connections 
   private final CopyOnWriteArrayList<T> sharedList;
   private final boolean weakThreadLocals;

   //To cache all connections used by a thread, equivalent to a quick reference, is an ArrayList of type ThreadLocal. 
   private final ThreadLocal<List<Object>> threadList;
   private final IBagStateListener listener;
   
   //The number of waiters currently obtaining connections. AtomicInteger, which is an increment object. A waiters WAITERS value greater than 0 means that a thread is currently fetching resources. 
   private final AtomicInteger waiters;
   private volatile boolean closed;

   //A 0-capacity fast delivery queue, of the SynchronousQueue type, is very useful 
   private final SynchronousQueue<T> handoffQueue;


   //In order to be able to operate lock-free, some variables need to be used to identify the current state. The abstract interface is as follows 
   public interface IConcurrentBagEntry
   {
      int STATE_NOT_IN_USE = 0;
      int STATE_IN_USE = 1;
      int STATE_REMOVED = -1;
      int STATE_RESERVED = -2;

      boolean compareAndSet(int expectState, int newState);
      void setState(int newState);
      int getState();
   }

   public interface IBagStateListener
   {
      void addBagItem(int waiting);
   }

   public ConcurrentBag(final IBagStateListener listener)
   {
      this.listener = listener;
      this.weakThreadLocals = useWeakThreadLocals();

      this.handoffQueue = new SynchronousQueue<>(true);
      this.waiters = new AtomicInteger();
      this.sharedList = new CopyOnWriteArrayList<>();
      if (weakThreadLocals) {
         this.threadList = ThreadLocal.withInitial(() -> new ArrayList<>(16));
      }
      else {
         this.threadList = ThreadLocal.withInitial(() -> new FastList<>(IConcurrentBagEntry.class, 16));
      }
   }

   /**
    * Core. to get the connection
    */
   public T borrow(long timeout, final TimeUnit timeUnit) throws InterruptedException
   {
      // Try the thread-local list first (If a thread executes very fast and uses a large number of connections, it can use ThreadLocal to get connection objects quickly without having to go into a large pool to get them)
      final var list = threadList.get();
      for (int i = list.size() - 1; i >= 0; i--) {
         final var entry = list.remove(i);
         @SuppressWarnings("unchecked")
         final T bagEntry = weakThreadLocals ? ((WeakReference<T>) entry).get() : (T) entry;
         if (bagEntry != null && bagEntry.compareAndSet(STATE_NOT_IN_USE, STATE_IN_USE)) {
            return bagEntry;
         }
      }

      // Otherwise, scan the shared list ... then poll the handoff queue
      final int waiting = waiters.incrementAndGet();
      try {

         //This code may be executed by a different thread, so thread safety must be considered. Since shardList is a thread-safe CopyOnWriteArrayList, which is suitable for read more write less scenarios, we can do the traversal directly

         for (T bagEntry : sharedList) {
            if (bagEntry.compareAndSet(STATE_NOT_IN_USE, STATE_IN_USE)) {
               // If we may have stolen another waiter's connection, request another bag add.
               if (waiting > 1) {
                  listener.addBagItem(waiting - 1);
               }
               return bagEntry;
            }
         }

         listener.addBagItem(waiting);

         timeout = timeUnit.toNanos(timeout);
         do {
            final var start = currentTime();
            final T bagEntry = handoffQueue.poll(timeout, NANOSECONDS);
            if (bagEntry == null || bagEntry.compareAndSet(STATE_NOT_IN_USE, STATE_IN_USE)) {
               return bagEntry;
            }

            timeout -= elapsedNanos(start);
         } while (timeout > 10_000);

         return null;
      }
      finally {
         waiters.decrementAndGet();
      }
   }

   /**
    * This method will return a borrowed object to the bag.
    */
   public void requite(final T bagEntry)
   {
      bagEntry.setState(STATE_NOT_IN_USE);

      for (var i = 0; waiters.get() > 0; i++) {
         if (bagEntry.getState() != STATE_NOT_IN_USE || handoffQueue.offer(bagEntry)) {
            return;
         }
         else if ((i & 0xff) == 0xff) {  //Because waiters () is available in real time, it is possible to keep your code greater than 0 for long periods of time, so that the code is looping and wasting CPU. The code attempts different levels of sleep
            parkNanos(MICROSECONDS.toNanos(10)); //one at 10ns every 255 waiters 
         }
         else {
            Thread.yield();  //one at yielding cpu time slices using yield
         }
      }

      final var threadLocalList = threadList.get();
      if (threadLocalList.size() < 50) {
         threadLocalList.add(weakThreadLocals ? new WeakReference<>(bagEntry) : bagEntry);
      }
   }


   ...

 
}

There are many knowledge points embedded within. Below, I list some key points you can delve into one by one.

  1. Use ThreadLocal to cache local resource references and reduce lock conflicts by using thread-encapsulated resources.
  2. Use the thread-safe CopyOnWriteArrayList, which supports more reads than writes, to cache all objects. This has minimal impact on read efficiency.
  3. Use AtomicInteger, based on CAS (Compare-and-Swap), to calculate the number of waiters. The lock-free operation makes computation faster.
  4. Use a zero-capacity SynchronousQueue for swapping, making object transfer quicker.
  5. Control status changes using the CAS primitive compareAndSet. This method is safe and efficient. Many core codes are designed this way.
  6. Use methods like park and yield in loops to avoid busy looping that can consume large amounts of CPU
  7. Understand the differences between offer, poll, peek, put, take, add, and remove methods in concurrent data structures and apply them flexibly.
  8. Use the volatile keyword in CAS when setting status. The use of volatile is a common optimization point.
  9. Understand the behavior of WeakReference weak references during the garbage collection process.

Multithreaded programming is profound and extensive. If you can master the above knowledge points, it will be of great help to the development of multithreaded concurrent issues in the future.

Good Luck on learing.

OK. now you can understand why HikariCP is one of the fastest connection pool systems available in Java.

Thanks for reading! If you like it or feel it helped pls click Applaud. Thanks :) Happy coding. See you next time.

Database
Programming
Spring Boot
Java
Coding
Recommended from ReadMedium
avatarPrasad Patare
Multi-Tenant Architecture using SpringBoot and PostgreSQL

Multi-Tenancy in Spring Boot Using PostgreSQL (Schema-per-Tenant)

5 min read
avatarJava Interview
Spring Says Goodbye to @Autowired: Here’s What to Use Instead

Yes, starting with Spring Boot 3 and Spring Framework 6, Spring has been encouraging constructor-based dependency injection over field…

3 min read
avatarSkilled Coder
Lesser Known Java Optimizations That Make a Big Difference

These minor refactors can drastically improved efficiency

6 min read
avatarUmadevi R
Unknown Java Facts That Will Surprise Even Experienced Developers

Java has been around for decades, yet it still holds some secrets that even seasoned developers might not know. While we often deal with…

2 min read
avatarAlexander Obregon
Java Memory Leaks: Detection and Prevention

Introduction

8 min read
avatarJeroen Rosenberg
Simplifying Hexagonal Architecture: How to Avoid Excessive Boilerplate

Hexagonal architecture, or Ports and Adapters, is a pattern that separates the core business logic of an application from its (external)…

8 min read