The Fundamental Knowledge of System Design — (6) — Buffer
The main purpose of the buffer is to carry out traffic shaping, to organize a large number of small-scale I/Os into a stable small number of large-scale I/Os, so as to reduce the number of responses. In other words, the buffer is used to balance the processing speed at both ends of the system. For example, you cannot write the hard disk with a little bit of data, but accumulate a certain amount of data and write it together in one block.
It is the sixth series of the fundamentals knowledge of system design. You can read my previous articles.
The storage structure of cache and buffer is different.
A buffer is a block of memory that stores data exchanged with external memory. CPU cannot communicate directly with the external memory. For the external memory, the read and write speed is much slower. The buffer is used to store data transmission between 2 devices with different speeds or different priorities. Hence, the buffer can reduce the waiting time for inter-process communication. When a device with a faster storage speed communicates with a device with a slow storage speed. To store data from the device with a slower storage speed, the system will store data in the buffer first, and then read the buffer data when the device with a faster storage speed reaches a certain level. At the same time, a device with a faster storage speed can do other tasks.
For example, Binary Terrain (BT) format is used to store elevation grids. BT download requires a long time to hang up, and the computer may run continuously for 24 hours, but the data downloaded by BT is fragmented, which is also reflected in the hard disk writing because the hard disk is a mechanical addressing device. Such fragmented writing will cause long-term high-load mechanical movement of the hard disk, resulting in premature aging and damage to the hard disk. Hence, the newly released BT software applies the concept of the buffer to the memory. The data was temporarily written into the buffer, up to a certain size (512M), and then written to the hard disk at one time. This method greatly reduces the high-load mechanical movement of the hard disk.
For example, if you watch a video online, the video player will preload a few seconds of video resources into the buffer at the beginning. The rate at which you consume the video is the normal playback of the video resource. Under such conditions, the video can be played smoothly. If the network speed is strong, more video resources will be downloaded and accumulated, until the buffer is full, temporarily stop loading, wait for part of the buffer to be emptied and start loading again. Even though the download speed of the video fluctuates, the video you watch is stable in terms of playback rate.
For example, when you have resources to write to the hard disk, the minimum writes size of the hard disk is often a block, usually 4KB. So, the 4KB of data will be stored in the buffer, then will be written to the hard disk, instead of preparing a byte of data each time. At this time, the disk consumes data at a rate of 4KB each time.
During the network connection process, the Transmission Control Protocol (TCP) connection maintained in the kernel may have a large amount of data written into the TCP buffer due to the network speed. It is because the application program corresponding to the socket may not be able to consume the data in time.
The cache is a storage that is between CPU and memory, increasing the access time by considering the read and write speed of CPU and memory, and allocating data based on the principle of locality. For example, for the spatial locality, the system will add data to the cache which is next to memory blocks. For temporal locality, the least frequently used (LFU) will be applied to remove the data from the cache.
Read buffer & Write Buffer
The data of the read buffer is not always valid but is generated in real-time. Whenever the buffer is full or the buffer is actively flushed, a read is triggered. For small data, this can reduce the number of reads. For large data, this amount of data read is normalized according to the buffer size. It means that the data fed to the buffer first must be read. it is an obvious sequential access feature.
The write buffer is the corresponding read buffer. For small data writing, it needs to fill up the write buffer. For large data, the big data will be divided into buffer sizes and written in batches. Writing 4K at a time is the most efficient for a certain device. Accumulate small data to 4K in the buffer, write-once or divide large data into 4K for each fragment, write multiple times.
The Various Buffering Techniques
Single buffer
- only one buffer is used
- The producer produces one block of data into the buffer. The consumer consumes the buffer. Once the buffer is empty again, the next block of data will be moved to the buffer.
- Most of the time, a connection always uses a buffer.
- The goal of a single buffer is to provide absoluteI/O consistency, ensure I/O performance, and reduce memory usage as much as possible.
Double buffer
- only two buffer is used
- The producer produces one block of data into buffer 1. While the consumer consumes buffer 2 simultaneously. So, there is no need to wait for filling the buffer. When buffer 1 is full, swap buffer 1 to buffer 2, and the consumer consumes the data of buffer 1.
- It is also known as buffer swapping
- It can reduce the lock time as much as possible because the producer and consumer do not operate on the same buffer. In a small fraction of the time, it is still possible for them to operate on the same buffer
- However, the complexity of the process gets increased. If the process performs rapid bursts of I/O, it may be deficient.
Circular buffer
- Only when more than 2 buffers are used.
- The collection of buffers can be called a circular buffer or a ring buffer
- The producer can only fill up to a buffer (n-i) while data in a buffer (i) is waiting to be consumed
- According to the Wikipedia “Circular buffer”, there are 3 characteristics such as the length of the buffer is fixed, first-in-first-out, and when removing elements to the buffer, the positions of other elements remain unchanged.
- The data transfer rate will increase using the circular buffer rather than the double buffering. It is suitable for the process that performs rapid bursts of I/O.
References
Disclaimer: As an amazon associate, I earn from qualifying purchase.
If you’ve found any of my articles helpful or useful then please consider throwing a coffee my way to help support my work or give me patronage😊, by using
Last but not least, if you are not a Medium Member yet and plan to become one, I kindly ask you to do so using the following link. I will receive a portion of your membership fee at no additional cost to you.
It is my first affiliate program, if you like to further enhance your system knowledge, you can click the links and buy the course. Honestly speaking, I will receive 20% of your course fees at no additional cost to you. You will have unlimited access to our courses. There is no time expiry and you will have access to all future updates free of cost.