Part 8 — Complete System Design Series

System Design Made Easy…

Welcome back peeps. In the last part ( links below) we covered in detail ( with examples) —

Projects Videos —

All the projects, data structures, SQL, algorithms, system design, Data Science and ML , Data Analytics, Data Engineering, , Implemented Data Science and ML projects, Implemented Data Engineering Projects, Implemented Deep Learning Projects, Implemented Machine Learning Ops Projects, Implemented Time Series Analysis and Forecasting Projects, Implemented Applied Machine Learning Projects, Implemented Tensorflow and Keras Projects, Implemented PyTorch Projects, Implemented Scikit Learn Projects, Implemented Big Data Projects, Implemented Cloud Machine Learning Projects, Implemented Neural Networks Projects, Implemented OpenCV Projects,Complete ML Research Papers Summarized, Implemented Data Analytics projects, Implemented Data Visualization Projects, Implemented Data Mining Projects, Implemented Natural Leaning Processing Projects, MLOps and Deep Learning, Applied Machine Learning with Projects Series, PyTorch with Projects Series, Tensorflow and Keras with Projects Series, Scikit Learn Series with Projects, Time Series Analysis and Forecasting with Projects Series, ML System Design Case Studies Series videos will be published on our youtube channel ( just launched).

Subscribe today!

Solved System Design Case Studies — In depth

Design Google Drive

Design Instagram

Design Quora

Design Flipkart

ML System Design

Design Tiny URL

Design Netflix

Design Messenger App

Design Twitter

Design Foursquare

Design Reddit

Design Amazon

Design Dropbox

Design URL Shortener

Design Youtube

Design API Rate Limiter

Design Web Crawler

Design Amazon Prime Video

Design Facebook’s Newsfeed

Design Yelp

Design Uber

Design Tinder

Design Tiktok

Design Whatsapp

Most Popular System Design Questions

Mega Compilation : Solved System Design Case studies

All the Complete System Design Series Parts —

1. System design basics

2. Horizontal and vertical scaling

3. Load balancing and Message queues

4. High level design and low level design, Consistent Hashing, Monolithic and Microservices architecture

5. Caching, Indexing, Proxies

6. Networking, How Browsers work, Content Network Delivery ( CDN)

7. Database Sharding, CAP Theorem, Database schema Design

8. Concurrency, API, Components + OOP + Abstraction

9. Estimation and Planning, Performance

10. Map Reduce, Patterns and Microservices

11. SQL vs NoSQL and Cloud

Moving forward, this is the part 8 of the system design series where we will be covering —

1. Concurrency
2. API
3. Components + OOP + Abstraction

Part 1 of this series can be found here —

Part 2 of this series can be found here —

Part 3 of this series can be found here —

Part 4 of this series can be found here —

Part 5 of this series can be found here —

Part 6 of this series can be found here —

Part 7 of this series can be found here —

And Most popular System Design Questions —

Let’s dive in!

Note : Please read System Design Important Terms you MUST know before reading this post.

Concurrency

Pasta Resto Case:

Now your pasta resto is growing exponentially and to make things work smoother you hire another manager who can delegate the work responsibilities efficiently and ask the workers-servers-chefs concurrently. It’s no longer divided by one man task instead whoever worker is free, you assign the task. All the workers/servers have access to the common areas ( kitchens, table areas, rooftops etc) and can access-modify-rewrite the details of any customer they have been tasked to serve by the manager concurrently.

System Design Analogy:

Taking the same analogy, concurrency is a very important concept in system design and a good understanding of concurrency goes a very long way.

So, what is concurrency?

In layman’s terms, concurrency is the process in which multiple computations/operations/process happen/execute in parallel/concurrently.

To understand concurrency, first you need to understand process and thread.

Process is nothing but an instance of a program which is executing independent of other processes.

Thread is a just part/segment of the process. A process can have multiple threads which interact with each other, perform different operations simultaneously and access the same shared resource.

Here is an example of how processes and threads can be implemented in a system using a Python programming language:

import multiprocessing
import threading
import time

def worker_process(num):
    """Worker process function"""
    print("Process: %d" % num)
    time.sleep(1)
    return

def worker_thread(num):
    """Worker thread function"""
    print("Thread: %d" % num)
    time.sleep(1)
    return

processes = []
for i in range(5):
    p = multiprocessing.Process(target=worker_process, args=(i,))
    processes.append(p)
    p.start()

threads = []
for i in range(5):
    t = threading.Thread(target=worker_thread, args=(i,))
    threads.append(t)
    t.start()

# Wait for all processes and threads to finish
for p in processes:
    p.join()

for t in threads:
    t.join()

print("Finished!")

In this example, two functions are defined: worker_process and worker_thread. The main program creates 5 processes and 5 threads, each calling the appropriate worker function with a different argument, and starts them. The main program then waits for all the processes and threads to finish using the join method.

This example demonstrates the difference between processes and threads in a system. Processes are independent units of execution that run in separate memory spaces, while threads are lighter-weight entities that run within the same process and share its memory space.

A good reference is —

There are two ways concurrency works —

1. Message passing — sub modules interact with each other through message passing ( via a communication channel)

2. Shared memory — sub modules interact with each other by updating the shared memory directly (i.e reading and writing shared objects in the memory)

Concurrency refers to the ability of a system to perform multiple tasks at the same time. Concurrency techniques are methods used to enable a system to handle multiple tasks simultaneously and efficiently. Some common concurrency techniques include:

1. Multi-threading: This technique allows a program to divide a task into smaller parts that can be executed simultaneously by multiple threads.
2. Asynchronous programming: This technique allows a program to continue executing other tasks while waiting for a task to complete, without blocking the program’s execution.
3. Event-driven programming: This technique allows a program to respond to external events, such as user input or network requests, in a non-blocking manner.
4. Process-based concurrency: This technique allows a system to run multiple processes simultaneously, each with its own memory space and execution context.
5. Actors Model: This is a concurrency model that allows a system to handle multiple tasks by creating multiple “actors” that can execute tasks independently and communicate with each other.

Multi-threading:

Multi-threading is a way of running multiple threads (smaller units of a process) concurrently within a single process.

Implementation in Python:

import threading

def print_numbers():
    for i in range(10):
        print(i)

def print_letters():
    for letter in 'abcdefghij':
        print(letter)

# Create two threads
t1 = threading.Thread(target=print_numbers)
t2 = threading.Thread(target=print_letters)

# Start the threads
t1.start()
t2.start()

# Wait for the threads to complete
t1.join()
t2.join()

In the above code, we have defined two functions, print_numbers and print_letters, which will print numbers and letters respectively. We have created two threads, t1 and t2, which will execute these functions concurrently. We have started the threads using the start() method and waited for them to complete using the join() method.

Async programming:

Async programming is a way of running tasks concurrently without blocking the main thread.

Implementation of async programming implementation in Python using the asyncio module:

import asyncio

async def print_numbers():
    for i in range(10):
        print(i)
        await asyncio.sleep(1)

async def print_letters():
    for letter in 'abcdefghij':
        print(letter)
        await asyncio.sleep(0.5)

# Create an event loop
loop = asyncio.get_event_loop()

# Run the tasks concurrently
tasks = [
    loop.create_task(print_numbers()),
    loop.create_task(print_letters())
]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

In the above code, we have defined two async functions, print_numbers and print_letters, which will print numbers and letters respectively. We have created an event loop using the asyncio.get_event_loop() method and run the tasks concurrently using the asyncio.wait() method. We have also used the asyncio.sleep() method to pause the execution of the tasks for a certain amount of time.

Event-driven programming:

Event-driven programming is a programming paradigm in which the flow of the program is determined by events such as user input, sensor data, or system events.

Implementation of event-driven programming implementation in Python using the tkinter module:

import tkinter as tk

def button_clicked():
    print('Button clicked')

# Create a GUI window
window = tk.Tk()

# Create a button
button = tk.Button(window, text='Click me', command=button_clicked)
button.pack()

# Start the event loop
window.mainloop()

In the above code, we have created a GUI window using the tkinter module and added a button to it. We have defined a function button_clicked which will be executed when the button is clicked. We have used the command parameter of the Button class to specify the function to be executed when the button is clicked. We have started the event loop using the mainloop() method of the Tk class.

Process-based concurrency:

Process-based concurrency is a way of running multiple processes concurrently within a single system.

Implementation in Python using the multiprocessing module:

import multiprocessing

def print_numbers():
    for i in range(10):
        print(i)

def print_letters():
    for letter in 'abcdefghij':
        print(letter)

# Create two processes
p1 = multiprocessing.Process(target=print_numbers)
p2 = multiprocessing.Process(target=print_letters)

# Start the processes
p1.start()
p2.start()

# Wait for the processes to complete
p1.join()
p2.join()

In the above code, we have defined two functions, print_numbers and print_letters, which will print numbers and letters respectively. We have created two processes, p1 and p2, which will execute these functions concurrently. We have started the processes using the start() method and waited for them to complete using the join() method.

Actor model:

The Actor model is a way of achieving concurrency by creating independent entities called actors, which communicate with each other through messages.

Implementation in Python using the pykka module:

import pykka

class Printer(pykka.ThreadingActor):
    def on_receive(self, message):
        if isinstance(message, int):
            print(message)
        else:
            raise ValueError('Invalid message')

# Create two printers
p1 = Printer.start().proxy()
p2 = Printer.start().proxy()

# Send messages to the printers
p1.tell(1)
p2.tell(2)

# Stop the printers
p1.stop()
p2.stop()

In the above code, we have defined an actor class Printer, which will print the messages it receives. We have created two printer actors, p1 and p2, which will execute the Printer class. We have sent messages to the printers using the tell() method and stopped the printers using the stop() method. We have used the proxy() method to create a proxy object to interact with the actors.

These techniques are used to handle multiple tasks at the same time and improve the overall performance of the system. The right concurrency technique to be used depends on the type of the problem and the system requirements.

Here is an example of how concurrency can be implemented in a system using a Python programming language:

import threading

def worker(num):
    """Thread worker function"""
    print("Worker: %d" % num)

threads = []
for i in range(5):
    t = threading.Thread(target=worker, args=(i,))
    threads.append(t)
    t.start()

# Wait for all threads to finish
for t in threads:
    t.join()

print("Finished!")

In this example, a function called worker is defined which will be executed in a separate thread. The main program creates 5 threads, each calling the worker function with a different argument, and starts them. The main program then waits for all the threads to finish using the join method.

This example demonstrates the basic concept of concurrency in a system, where multiple tasks are executed in parallel. By using threads, the system can handle multiple tasks simultaneously, improving its performance and responsiveness.

In a concurrent system, tasks are executed in parallel, allowing multiple tasks to be processed at the same time. To handle concurrency, the system needs to have the following components:

1. Task Scheduling: Task scheduling is responsible for deciding which tasks to run and when. In a concurrent system, tasks are scheduled based on priority, availability of resources, and other factors.
2. Task Management: Task management is responsible for creating and managing tasks, including their execution and synchronization. Task management ensures that tasks are executed efficiently and that the system remains in a consistent state even when multiple tasks are executed simultaneously.
3. Resource Management: Resource management is responsible for allocating and managing system resources, such as memory, CPU, and disk I/O, to ensure that tasks have access to the resources they need to run efficiently.
4. Synchronization: Synchronization is responsible for coordinating the execution of tasks to ensure that they access shared resources in a consistent and safe manner. This includes techniques such as mutual exclusion, locks, and semaphores.
5. Deadlock Detection and Recovery: Deadlocks can occur when multiple tasks wait for each other to release resources, resulting in a situation where no tasks can proceed. Deadlock detection and recovery is responsible for identifying and resolving deadlocks to ensure that the system remains operational.

In a concurrent system, careful consideration must be given to the design of the system to ensure that it can handle multiple tasks simultaneously while maintaining data consistency, performance, and reliability. To achieve this, system designers often employ techniques such as locking, semaphores, and other synchronization mechanisms to ensure that tasks access shared resources in a safe and consistent manner. Additionally, load balancing and task scheduling algorithms are used to distribute tasks and resources efficiently to ensure that the system remains scalable and responsive.

What are the advantages of implementing concurrency?

1. Efficient Resource Utilization

2. Better performance

3. Sharing resources and using them optimally

4. Better turn around time

While there are many advantages of concurrency, some of the major drawbacks are — deadlocks, racing for resources, starvation and blocking.

API

Pasta Resto Case:

Lets say, one of the customer really liked your calavita rigatoni pasta and came to you if you can share the recipe ( ofcourse! for a cost). You like to keep your customer happy and since this customer happens to a food blogger too — you think its a win-win for both — The customer will get the recipe and you will get new customers via his food blog.

So, you decide to share the recipe but not the full recipe ( as it’s your trade secret). So, you give the customer only the bullet points in the form of a note and whenever he wants to cooks rigatoni, he can refer to that note and to elaborate more on the bullet points he can call the owner ( pay some more money for each time).

This way, the customer is able to use the owner’s expertise and the owner can make some good money by selling recipe’s in the form of notes ( covering only bullet points)

System Design Analogy:

Taking the same analogy, API is one of the most important concepts in system design and you MUST know what it is.

So what is API?

API is an acronym for application programming interface which provides a way to two or more programs to communicate, work together despite different configurations, architectures, resources etc

APIs make the internet so powerful and systems easy to work with. Through API you can make requests, designate end points, get responses, provide a valuable service, simplify the design and many other zillion things that you can do with API’s.

API is a set of protocols, routines, and tools for building software and applications. APIs specify how software components should interact and APIs allow for communication between different systems.

An API defines a set of rules and conventions for building and interacting with software and applications. It specifies how software components should interact and APIs allow for communication between different systems.

APIs are used to enable communication between different software systems. For example, a weather app on a smartphone might use an API to request and retrieve data from a weather service. The API defines how the app should request the data (using specific parameters and formatting) and how the weather service should respond (with specific data in a specific format).

APIs can be classified into several types, such as:

• Web-based APIs that use HTTP protocol to GET, PUT, POST and DELETE data
• Library-based APIs that allow developers to include and use a pre-written library in their code
• Operating system-based APIs that allow access to the capabilities of an operating system

Here’s a brief overview of how an API works internally:

1. Request-Response cycle: An API operates through a request-response cycle. A client sends a request to an API endpoint and the API sends back a response. The request contains information about the desired operation and any necessary data, while the response contains the result of the operation or an error message if something went wrong.
2. Endpoints: An API defines endpoints, or specific locations for accessing resources or executing operations. Endpoints are specified by a unique URL, and the type of request (e.g., GET, POST, PUT, DELETE) indicates the type of operation to be performed.
3. Routing: When a client sends a request to an API endpoint, the API’s routing layer maps the request to the appropriate handler function. This function processes the request and prepares the response to be returned to the client.
4. Data serialization: The data exchanged between the client and the API is usually serialized and deserialized, meaning that it is converted from its native data format into a format suitable for transmission over the network, and vice versa. This is usually done using data formats such as JSON or XML.
5. Business logic: The API’s business logic layer implements the core functionality of the API. This layer accesses databases, performs computations, and generates responses based on the request received from the client.
6. Security: The API should also implement security measures to protect against potential threats such as unauthorized access, data tampering, and denial of service attacks. This can be achieved using techniques such as authentication, encryption, and rate limiting.

In summary, an API is a set of instructions that allow different systems to communicate and share data. It defines how the systems should request and receive data, making it easier for developers to access and use that data in their own applications.

To make a API call, you can use common requests methods —

PUT: To edit an existing resource

GET: To fetch a resource

POST: To create a new resource

DELETE: To delete the resource

Here’s an example of how to implement an API using the Flask framework in Python:

• Install Flask: To implement an API using Flask, you’ll first need to install the Flask library. You can do this by running the following command in your terminal:

pip install Flask

• Create a Flask application: Next, you’ll need to create a Flask application. You can do this by creating a new Python file and adding the following code:

from flask import Flask, jsonify

app = Flask(__name__)

@app.route("/")
def index():
    return "Welcome to my API!"

if __name__ == '__main__':
    app.run()

This code creates a Flask application and defines a single endpoint at the root URL (/) that returns a simple "Welcome to my API!" message.

• Define an endpoint: To define an endpoint, you use the @app.route decorator and specify the URL for the endpoint. For example, the following code defines an endpoint for retrieving a list of users:

@app.route("/users", methods=['GET'])
def get_users():
    users = [
        {"id": 1, "name": "John Doe"},
        {"id": 2, "name": "Jane Doe"},
    ]
    return jsonify(users)

This endpoint returns a list of dictionaries, representing users, in JSON format. The jsonify function is used to convert the data into a JSON response.

• Start the server: To start the server, run the following command in your terminal:

export FLASK_APP=api.py
flask run

This will start the Flask development server on http://localhost:5000. You can now send requests to the API and receive responses.

API gateway ( reverse proxy) is used to collect requests and delegate/route them accordingly ( as shown in the pic above).

Read more about proxies and gateways —

API design is MUST know thing and we will be covering it detail as we design different systems subsequently.

Component Design + OOP + Abstraction

Pasta Resto Case:

To run a successful pasta restaurant, you as an owner have decided that you will make each department in your resto as one single component and designate one person ( leader/manager of each department) who will answer all the questions on everybody’s behalf to address any issue/share anything. That’s how you can run the resto smoothly and at the same time address all the queries/feedbacks within a day itself.

To summarize, you will now treat every department as an object with their own data and methods.

System Design Analogy:

Taking the same analogy, every module in the system design should be conceptualized and treated as a component ( which can function autonomously) and can be reused.

Components in the system design are building blocks designed to coordinate, cooperate, reuse and work well with other components of the same/different systems. They can be as simple as visual components or internal components/backend components

At some level, what ever components you build in your system should follow OOP ( Object oriented programming) principles. Everything in OOP is an object.

Class —To bind together object’s data and behaviour(method)

Inheritance — To create sub classes from the existing class by deriving/inheriting the attributes and methods of the existing class

Polymorphism — To carry out same operation differently in different classes of the objects i.e take many forms

Abstraction — To only use necessary/essential characteristics for attributes and methods to define an object.

Encapsulation — To hide the information ( data, attributes, methods) into a single entity.

This is a great post by Stanford to understand abstraction.

Here’s a code example in Python that demonstrates the implementation of component design, object-oriented programming (OOP), and abstraction:

class Car:
    def __init__(self, make, model, year):
        self.make = make
        self.model = model
        self.year = year
        
    def start(self):
        print(f"{self.make} {self.model} ({self.year}) started")
        
    def stop(self):
        print(f"{self.make} {self.model} ({self.year}) stopped")
        
    def drive(self):
        print(f"{self.make} {self.model} ({self.year}) is driving")

class ElectricCar(Car):
    def __init__(self, make, model, year):
        super().__init__(make, model, year)
        
    def start(self):
        print(f"{self.make} {self.model} ({self.year}) started silently")
        
    def drive(self):
        print(f"{self.make} {self.model} ({self.year}) is silently driving")

my_car = ElectricCar("Tesla", "Model S", 2022)
my_car.start()  # Tesla Model S (2022) started silently
my_car.drive()  # Tesla Model S (2022) is silently driving
my_car.stop()  # Tesla Model S (2022) stopped

In this example, the Car class defines the basic behavior of a car, such as starting, stopping, and driving. The ElectricCar class is a subclass of the Car class and represents a specific type of car: an electric car. This class uses inheritance and polymorphism to override the methods start and drive of the parent class, to reflect the specific behavior of an electric car. This is an example of component design, as we are breaking down the problem into smaller components that can be reused and modified independently.

The ElectricCar class uses the super() function to call the constructor of the parent class, which is an example of OOP. This allows us to define the behavior of an electric car in terms of the behavior of a more general car, making it easier to understand and maintain the code.

The abstraction is demonstrated by the way we define the car’s behavior without specifying how it’s actually implemented. For example, the drive method in the Car class just specifies that the car is driving, without specifying how the driving is actually accomplished. This makes the code more flexible, as we can change the implementation details without affecting the code that uses the car's behavior.

In Object-Oriented Programming (OOP), abstraction is the process of hiding the implementation details of an object and only exposing the essential features. It is one of the four main principles of OOP, along with encapsulation, inheritance, and polymorphism.

• Abstraction allows developers to focus on the functionality of an object, rather than its internal workings. It helps to simplify the design of complex systems by breaking them down into smaller, more manageable parts.
• Abstraction can be achieved through the use of classes and interfaces. A class is an abstraction of an object, defining its properties and methods. An interface is an abstraction of a class, defining a contract for the behavior of the class without specifying the implementation details.
• For example, a car class can have properties such as “engine” and “wheels” and methods such as “start()” and “drive()”. These details are abstracted away from the user of the car class, who only needs to know that they can start and drive the car, without needing to know the specific details of how the engine and wheels work.

In summary, abstraction in OOP is the process of hiding the implementation details of an object and only exposing the essential features. It allows developers to focus on the functionality of an object, rather than its internal workings, and helps to simplify the design of complex systems.

More on Concurrency, API and Components + OOP + Abstraction —

Concurrency:

Concurrency refers to the ability of a system to execute multiple tasks or processes simultaneously, allowing for efficient utilization of resources and improved system performance. It is crucial in system design to handle concurrent user requests, maximize throughput, and ensure responsiveness.

Threads vs. processes: In concurrent programming, a thread represents an independent flow of execution within a process. Threads share the same memory space and resources of a process, enabling efficient communication and coordination. Processes, on the other hand, are independent instances of a program that run in separate memory spaces. They have their own set of resources and do not share memory directly. Threads provide lightweight concurrency with less overhead compared to processes.

Thread synchronization: Thread synchronization is essential to coordinate access to shared resources and prevent data inconsistencies or race conditions. Various synchronization techniques can be used:

Locks: Implementing locks, such as mutexes and semaphores, allows threads to acquire exclusive access to shared resources. This prevents concurrent access and ensures data integrity.

import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;

Lock lock = new ReentrantLock();

void performTask() {
    lock.lock();
    try {
        // Critical section - Access shared resource
    } finally {
        lock.unlock();
    }
}

Semaphores: Semaphores control access to a limited number of resources. Threads can acquire and release semaphores to manage access to shared resources.

import java.util.concurrent.Semaphore;

Semaphore semaphore = new Semaphore(5);

void performTask() {
    try {
        semaphore.acquire();
        // Critical section - Access shared resource
    } catch (InterruptedException e) {
        // Handle interruption
    } finally {
        semaphore.release();
    }
}

Concurrent data structures: Concurrency-safe data structures are designed to handle concurrent access and modifications by multiple threads. Examples include concurrent queues, hash tables, and skip lists.

import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentSkipListSet;

ConcurrentLinkedQueue<String> queue = new ConcurrentLinkedQueue<>();
ConcurrentHashMap<String, Integer> map = new ConcurrentHashMap<>();
ConcurrentSkipListSet<Integer> set = new ConcurrentSkipListSet<>();

Parallel programming models: Parallel programming models enable the execution of tasks in parallel to take advantage of multicore processors and distributed systems. Some common models include:

Task parallelism: Breaking a problem into smaller tasks that can be executed concurrently, often utilizing thread pools or task schedulers.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

ExecutorService executorService = Executors.newFixedThreadPool(4);

executorService.submit(() -> {
    // Task 1
});

executorService.submit(() -> {
    // Task 2
});

executorService.shutdown();

Data parallelism: Distributing data across multiple processing units and performing operations on subsets of the data simultaneously.

import java.util.Arrays;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.RecursiveAction;

class DataParallelTask extends RecursiveAction {
    private static final int THRESHOLD = 1000;
    private int[] data;
    private int start;
    private int end;

    public DataParallelTask(int[] data, int start, int end) {
        this.data = data;
        this.start = start;
        this.end = end;
    }

    @Override
    protected void compute() {
        if (end - start <= THRESHOLD) {
            // Perform computation on data subset
        } else {
            int mid = (start + end) / 2;
            invokeAll(
                new DataParallelTask(data, start, mid) 

Design patterns: Design patterns provide reusable solutions to common design problems and promote modularity, flexibility, and extensibility in component-based systems. Here are some commonly used design patterns:

Singleton: Ensures that only one instance of a class is created and provides a global access point to it.

public class Singleton {
    private static Singleton instance;

    private Singleton() {
        // Private constructor to prevent instantiation
    }

    public static Singleton getInstance() {
        if (instance == null) {
            synchronized (Singleton.class) {
                if (instance == null) {
                    instance = new Singleton();
                }
            }
        }
        return instance;
    }
}

Factory: Provides an interface for creating objects, allowing subclasses or implementing classes to decide which concrete class to instantiate.

public interface Animal {
    void sound();
}

public class Dog implements Animal {
    @Override
    public void sound() {
        System.out.println("Woof!");
    }
}

public class Cat implements Animal {
    @Override
    public void sound() {
        System.out.println("Meow!");
    }
}

public class AnimalFactory {
    public Animal createAnimal(String type) {
        if ("dog".equalsIgnoreCase(type)) {
            return new Dog();
        } else if ("cat".equalsIgnoreCase(type)) {
            return new Cat();
        }
        throw new IllegalArgumentException("Invalid animal type: " + type);
    }
}

Observer: Defines a one-to-many dependency between objects, so that when one object changes its state, all its dependents are notified and updated automatically.

import java.util.ArrayList;
import java.util.List;

public interface Observer {
    void update();
}

public interface Subject {
    void registerObserver(Observer observer);
    void removeObserver(Observer observer);
    void notifyObservers();
}

public class ConcreteSubject implements Subject {
    private List<Observer> observers = new ArrayList<>();

    @Override
    public void registerObserver(Observer observer) {
        observers.add(observer);
    }

    @Override
    public void removeObserver(Observer observer) {
        observers.remove(observer);
    }

    @Override
    public void notifyObservers() {
        for (Observer observer : observers) {
            observer.update();
        }
    }
}

public class ConcreteObserver implements Observer {
    @Override
    public void update() {
        System.out.println("Observer is notified and updated.");
    }
}

Component-based architecture: Component-based architecture promotes the construction of systems from reusable, self-contained software components. These components encapsulate functionality, have well-defined interfaces, and can be independently developed, tested, and deployed.

Abstraction levels: Abstraction levels define the different layers of complexity and detail in a system. High-level abstraction focuses on overall architecture and system behavior, while lower-level abstraction deals with implementation details. This allows for modular design, easy maintenance, and loose coupling between components.

Dependency injection: Dependency injection is a technique where the dependencies of a component are provided from the outside rather than being created within the component itself. This promotes reusability, testability, and modularity.

public class UserService {
    private UserRepository userRepository;

    public UserService(UserRepository userRepository) {
        this.userRepository = userRepository;
    }
    // ...
}

Interface design: Interface design involves defining clear and intuitive interfaces between components to facilitate communication and collaboration. Good interface design focuses on simplicity, clarity, and consistency.

Component lifecycle management: Managing the lifecycle of components involves initialization, configuration, state management, and cleanup. This ensures proper handling of resources, initialization of dependencies, and graceful shutdown of components.

// Component: Calculator
public interface Calculator {
    int add(int a, int b);
    int subtract(int a, int b);
}

// Component Implementation: BasicCalculator
public class BasicCalculator implements Calculator {
    @Override
    public int add(int a, int b) {
        return a + b;
    }

    @Override
    public int subtract(int a, int b) {
        return a - b;
    }
}

// Component Implementation: AdvancedCalculator
public class AdvancedCalculator implements Calculator {
    @Override
    public int add(int a, int b) {
        return a + b;
    }

    @Override
    public int subtract(int a, int b) {
        return a - b;
    }

    public int multiply(int a, int b) {
        return a * b;
    }
}

// Usage of Components
public class Main {
    public static void main(String[] args) {
        Calculator calculator = new BasicCalculator();
        int sum = calculator.add(5, 3);
        int difference = calculator.subtract(5, 3);
        System.out.println("Sum: " + sum);
        System.out.println("Difference: " + difference);
    }
}

API:

APIs (Application Programming Interfaces) are sets of rules and protocols that define how software components should interact with each other. APIs enable communication and data exchange between different software systems, components, or services.

Types of APIs: There are various types of APIs, including RESTful APIs, SOAP APIs, GraphQL APIs, and WebSockets. RESTful APIs use HTTP methods to perform CRUD (Create, Read, Update, Delete) operations on resources. SOAP APIs use the SOAP protocol for communication and provide more extensive functionality.

API documentation: API documentation is crucial for developers who want to understand and utilize an API effectively. It provides comprehensive information about the API’s functionality, endpoints, parameters, request and response formats, authentication requirements, error handling, and usage examples. Well-documented APIs make it easier for developers to integrate the API into their applications and ensure proper usage.

API security: API security is essential to protect sensitive data and prevent unauthorized access or malicious activities. Common security concerns in API design include authentication, authorization, rate limiting, and data protection. Implementing secure authentication mechanisms such as OAuth or API keys, implementing proper authorization checks, setting rate limits to prevent abuse, and encrypting data in transit and at rest are essential practices for ensuring API security.

API testing and monitoring: Testing APIs is crucial to ensure their functionality, performance, and reliability. Unit testing, integration testing, and load testing are commonly used strategies for testing APIs. Additionally, monitoring API performance and usage is important to identify bottlenecks, track response times, detect errors or anomalies, and ensure optimal API performance. Tools like Postman, JUnit, and monitoring systems like Prometheus can be used for API testing and monitoring.

API versioning and backward compatibility: As APIs evolve over time, it is important to handle changes in a backward-compatible manner to avoid breaking existing integrations. API versioning allows the introduction of new features or modifications while maintaining backward compatibility with previous versions. Techniques like versioning through URL, headers, or query parameters can be employed to manage API versioning effectively.

API best practices: Adhering to best practices is crucial for designing effective and developer-friendly APIs. Some best practices include adhering to RESTful principles, using meaningful resource naming, providing appropriate status codes, leveraging caching mechanisms to improve performance, using descriptive error messages, and providing clear and concise documentation. Following these practices ensures consistency, usability, and scalability of the API.

// Required dependencies
const express = require('express');
const bodyParser = require('body-parser');

// Create Express app
const app = express();
app.use(bodyParser.json());

// Mock data for books
let books = [
  { id: 1, title: 'Book 1', author: 'Author 1' },
  { id: 2, title: 'Book 2', author: 'Author 2' },
  { id: 3, title: 'Book 3', author: 'Author 3' }
];

// GET /books - Retrieve all books
app.get('/books', (req, res) => {
  res.json(books);
});

// GET /books/:id - Retrieve a specific book
app.get('/books/:id', (req, res) => {
  const id = parseInt(req.params.id);
  const book = books.find((book) => book.id === id);

  if (book) {
    res.json(book);
  } else {
    res.status(404).json({ error: 'Book not found' });
  }
});

// POST /books - Create a new book
app.post('/books', (req, res) => {
  const { title, author } = req.body;

  if (!title || !author) {
    res.status(400).json({ error: 'Title and author are required' });
  } else {
    const id = books.length + 1;
    const newBook = { id, title, author };
    books.push(newBook);
    res.status(201).json(newBook);
  }
});

// PUT /books/:id - Update an existing book
app.put('/books/:id', (req, res) => {
  const id = parseInt(req.params.id);
  const { title, author } = req.body;
  const book = books.find((book) => book.id === id);

  if (!book) {
    res.status(404).json({ error: 'Book not found' });
  } else if (!title || !author) {
    res.status(400).json({ error: 'Title and author are required' });
  } else {
    book.title = title;
    book.author = author;
    res.json(book);
  }
});

// DELETE /books/:id - Delete a book
app.delete('/books/:id', (req, res) => {
  const id = parseInt(req.params.id);
  const index = books.findIndex((book) => book.id === id);

  if (index === -1) {
    res.status(404).json({ error: 'Book not found' });
  } else {
    const deletedBook = books.splice(index, 1);
    res.json(deletedBook[0]);
  }
});

// Start the server
app.listen(3000, () => {
  console.log('Server running on port 3000');
});

That’s it for now!

Part 9 of Complete System Design Series

Keep learning and coding :)

Day 1 : SQL Basics and Kick start of Advanced SQL Series

Day 2 : SQL Basics, Query Structure, Built In functions Conditions

Day 3 : Most Important Commands, Joins and Filters

Day 4 : Set Theory Operations, Stored Procedures and CASE statements in SQL

Day 5 : Wildcards, Aggregation and Sequences in SQL

Day 6 : Subqueries, Group by, order by and Having clauses in SQL and Analytical Functions

Day 7 : Window Functions, Grouping Sets and Constraints in SQL

Day 8 : BigQuery Basics, SELECT, FROM, WHERE and Date and Extract in BigQuery

Day 9 : Common Expression Table, UNNEST Clause, SQL vs NoSQL Databases

Day 10 : Triggers, Pivot and Cursors in SQL

Day 11 : Views, Indexes and Auto Increment in SQL

Day 12 : Query optimizations, Performance tuning in SQL

Day 13 : Introduction to MySQL, PostgreSQL and Mongo DB, Comparison between MySQL and PostgreSQL and Mongo DB, Introduction to SQL and NoSQL Databases

Day 14 : MySQL in Depth

Day 15 : PostgreSQL inDepth

Anyways, For Day 15 of 15 days of Advanced SQL, we will cover —

PostgreSQL inDepth

Github for Advanced SQL that you can follow —

All the projects, data structures, algorithms, system design, Data Science and ML, Data Engineering, MLOps and Deep Learning videos will be published on our youtube channel ( just launched).

Subscribe today!

System Design Case Studies — In Depth

Design Instagram

Design Messenger App

Design Twitter

Design URL Shortener

Design Dropbox

Design Youtube

Design API Rate Limiter

Design Web Crawler

Design Facebook’s Newsfeed

Design Yelp

Design Uber

Design Tinder

Design Tiktok

Design Whatsapp

Most Popular System Design Questions

Mega Compilation : Solved System Design Case studies

Complete Data Structures and Algorithm Series

Complexity Analysis

Backtracking

Sliding Window

Greedy Technique

Two pointer Technique

Arrays

Linked List

Strings

Stack

Queues

Hash Table/Hashing

Binary Search

1- D Dynamic Programming

Divide and Conquer Technique

Recursion

Github —

Some of the other best Series —

30 Days of Natural Language Processing ( NLP) Series

60 days of Data Science and ML Series with projects

30 days of Data Structures and Algorithms and System Design Simplified

60 Days of Deep Learning with Projects Series

100 days : Your Data Science and Machine Learning Degree Series with projects

23 Data Science Techniques You Should Know

Tech Interview Series — Curated List of coding questions

Complete System Design with most popular Questions Series

Complete Data Visualization and Pre-processing Series with projects

Complete Python Series with Projects

Complete Advanced Python Series with Projects

Kaggle Best Notebooks that will teach you the most

Complete Developers Guide to Git

All the Data Science and Machine Learning Resources

210 Machine Learning Projects

30 days of Machine Learning Ops

Tech Newsletter —

If you are interested, you can join my newsletter through which I send tech interview tips, techniques, patterns, hacks — Software Development, ML, Data Science, Startups and Technology projects to more than 30K readers. You can subscribe to Tech Brew :

For Python Projects —

For complete 60 days of Data Science and ML : Day 1 — Day 60 : Quick Recap of 60 days of Data Science and ML

Follow for more updates. Stay tuned and keep coding!

For other projects, tune to —

Build Machine Learning Pipelines( With Code)

Recurrent Neural Network with Keras

Clustering Geolocation Data in Python using DBSCAN and K-Means

Facial Expression Recognition using Keras

Hyperparameter Tuning with Keras Tuner

Custom Layers in Keras