System Design Made Easy — Quick Recap of Complete System Design Series
Connect the dots…
Welcome back peeps! This post is a quick roundup of all the things that I have covered so far in the system design made easy series.
Projects Videos —
All the projects, data structures, SQL, algorithms, system design, Data Science and ML , Data Analytics, Data Engineering, , Implemented Data Science and ML projects, Implemented Data Engineering Projects, Implemented Deep Learning Projects, Implemented Machine Learning Ops Projects, Implemented Time Series Analysis and Forecasting Projects, Implemented Applied Machine Learning Projects, Implemented Tensorflow and Keras Projects, Implemented PyTorch Projects, Implemented Scikit Learn Projects, Implemented Big Data Projects, Implemented Cloud Machine Learning Projects, Implemented Neural Networks Projects, Implemented OpenCV Projects,Complete ML Research Papers Summarized, Implemented Data Analytics projects, Implemented Data Visualization Projects, Implemented Data Mining Projects, Implemented Natural Leaning Processing Projects, MLOps and Deep Learning, Applied Machine Learning with Projects Series, PyTorch with Projects Series, Tensorflow and Keras with Projects Series, Scikit Learn Series with Projects, Time Series Analysis and Forecasting with Projects Series, ML System Design Case Studies Series videos will be published on our youtube channel ( just launched).
Subscribe today!
System Design Case Studies — In Depth
Design Instagram
Design Messenger App
Design Twitter
Design URL Shortener
Design Dropbox
Design Youtube
Design API Rate Limiter
Design Web Crawler
Design Facebook’s Newsfeed
Most Popular System Design Questions
Mega Compilation : Solved System Design Case studies
Complete Data Structures and Algorithm Series
Github —
All the Complete System Design Series Parts —
6. Networking, How Browsers work, Content Network Delivery ( CDN)
Github —
For complete 60 days of Data Science and ML : Day 1 — Day 60 : Quick Recap of 60 days of Data Science and ML
Highly Recommended Data Science and Machine Learning Courses that you MUST take ( with certificate) —
Find best data science and data engineering courses here
Find best Machine Learning and Deep Learning courses here
Let’s dive in!
Part 1 of System Design Made Easy Series
In the part 1, we covered what and why of System Design and the important topics that you should know. System Design is mostly an open ended concept and most of the questions can be answered in different degrees and aptitudes. In layman’s language, system design is about —
Architecture + Data + Applications
Learn what is system design and why it’s important, how they work and how to use them in your system design interviews in part 1( link below) —
Part 2 of System Design Made Easy Series
In the part 2, we covered —
System design basics
Horizontal and Vertical Scaling with an example
In technical words, scalability is a the technique/process of adding/removing infrastructure/resources required by applications to better serve/accommodate increased/decreased demand/growth. We also covered the tradeoffs of horizontal and vertical scaling.
Learn why good understanding of system design basics and horizontal and vertical scaling are important, how they work and how to use them in your system design interviews in part 2( link below) —
Part 3 of System Design Made Easy Series
In the part 3, we covered system design’s most important concepts —
Load Balancing
Message Queues
Load balancing is a technique of distributing tasks over a set of servers/machines to improve the performance, throughput, high availability, redundancy and reliability of the system. Not just it enables horizontal scaling but also dynamic resizing/scaling.
Message queues are nothing bit temporary buffers placed between users/applications and servers to store the message requests and process them in FIFO order asynchronously until the requests/messages are delivered to the desired server.
We also covered the tradeoffs of the different techniques.
Learn why load balancing and message queues are important, how they work and how to use them in your system design interviews in part 3( link below) —
Part 4 of System Design Made Easy Series
In the part 4, we covered
High level design and Low level design
Monolithic and microservices architecture and which one to choose and when?
Consistent Hashing
High level Design (HLD) describes the overall architecture of the application and covers functionality of each module of the system very briefly. LLD details the functional logic of the each module in the system.
Monolithic architecture — consists of single code base with multiple modules and it’s easier and faster to deploy. Microservices architecture — consists of individual service units with each service being responsible for exactly one functionality. It’s relatively complex and time taking to deploy.
Consistent hashing is a technique to divide keys/data between multiple servers/machines using a hash function ( key — value).
Learn why high level design and low level design, Monolith and microservices architecture and consistent hashing are important, how they work and how to use them in your system design interviews in part 4 ( link below) —
Part 5 of System Design Made Easy Series
In the part 5, we covered —
Caching
Indexing
Proxies
Caching is a technique which is based on the principal of locality — stores the copies of most frequently used/accessed data in a small and faster memory to improve Data retrieval times, Compute costs, User Experience and Throughput.
Indexing helps in Improving the speed of data access/retrieval, Reducing the number of expensive I/O operations, Providing better organization and management of multilevel data records.
Proxies play an important role of coordinating user requests, handling concurrent requests, filtering user request, transforming user requests by adding an additional layer of encryption or header information or compression information and then forwarding the user request to the server.
Learn why Caching, Indexing and Proxies are important, how they work and how to use them in your system design interviews in part 5( link below) —
Part 6 of System Design Made Easy Series
In the part 6, we covered —
Networking
How Browsers work
Content Network Delivery ( CDN)
Networking is nothing but interconnected devices that can exchange data-messages and share resources amongst themselves/with outside world based in the system protocols/rules, technologies and algorithms that govern these devices inner workings.
Browsers are used to present the website/resource you would like to visit say, for example google.com by sending the request to the server and displaying it on their browser window.
Content Network Delivery caters to the users by serving their requests by quickly transferring the data back and forth.
Learn why networking, browsers working and CDN are important, how they work and how to use them in your system design interviews in part 6( link below) —
Part 7 of System Design Made Easy Series
In the part 7, we covered —
Database Sharding
CAP Theorem
Database schema Design
Sharding is the technique to database partitioning that separates large and complex databases into smaller, faster and distributed databases for higher throughput operations.
CAP theorem lets you determine how you want to handle your distributed databases with there is possibility of inconsistencies, unavailability and connection errors/failures/outrage.
Database schema Design lets you organize data into separate entities and establish and organize the relationships between different entities.
Learn why database sharding, CAP theorem and Database Schema Design are important, how they work and how to use them in your system design interviews in part 7 ( link below) —
Part 8 of System Design Made Easy Series
In the part 8 , we covered —
Concurrency
API
Components + OOP + Abstraction
Concurrency is the process in which multiple computations/operations/process happen/execute in parallel/concurrently.
API is an acronym for application programming interface which provides a way to two or more programs to communicate, work together despite different configurations, architectures, resources etc
Components in the system design are building blocks designed to coordinate, cooperate, reuse and work well with other components of the same/different systems. They can be as simple as visual components or internal components/backend components.
Learn why concurrency, API and Components + OOP + Abstraction are important, how they work and how to use them in your system design interviews in part 8 (link below)—
Part 9 of System Design Made Easy Series
In the part 9 , we covered —
Planning and Estimation
Performance
Planning and estimation( numbers) and performance are very important concepts ( concept that you should be able to demonstrate well when asked).
Learn why Planning, estimation( numbers) and performance are important, how they work and how to use them in your system design interviews in part 9 (link below) —
Part 10 of System Design Made Easy Series
In the part 10, we covered —
Map Reduce
Patterns and Microservices
In system design, map reduce ( Hadoop systems) is a batch processing technique in which the engine takes huge amounts of data, processes ( map and reduce) and gives the output.
In system design, microservices architecture is used to build enterprise level applications which helps in structuring the whole application as a collection of tiny autonomous, self contained services for each task ( service) that you want/are allowed to perform.
Learn why map reduce, patterns and microservices are important, how they work and how to use them in your system design interviews in part 10 (link below) —
Part 11 of System Design Made Easy Series
In the part 11, we covered —
SQL vs NoSQL
Cloud
SQL databases have predefined schema and the data is organized/displayed in the form of tables. These databases use SQL ( Structured Query Language) to define, manipulate, update the data.
NoSQL databases on the other side, have no predefined schema which adds to more flexibility to use the formats that best suits the data — Work with graphs, column-oriented data, key-value and documents etc. They are generally preferred for hierarchical data, graphs ( e.g. social network) and to work with large data.
The cloud computing services are used to address the scalability, availability, cost management and disaster recovery etc of the application as the user base grows exponentially.
Learn why SQL vs NoSQL and Cloud are important, how they work and how to use them in your system design interviews in part 11 (link below) —
Most Popular System Design Questions — Mega Compilation
In this post. we covered the most popular/important system design questions that you should practice to build a thorough understanding of how large systems are designed.
Coming up next : Solution to Most popular system design Questions
Keep learning and coding :)
Day 2 : SQL Basics, Query Structure, Built In functions Conditions
Day 4 : Set Theory Operations, Stored Procedures and CASE statements in SQL
Day 6 : Subqueries, Group by, order by and Having clauses in SQL and Analytical Functions
Day 7 : Window Functions, Grouping Sets and Constraints in SQL
Day 8 : BigQuery Basics, SELECT, FROM, WHERE and Date and Extract in BigQuery
Day 9 : Common Expression Table, UNNEST Clause, SQL vs NoSQL Databases
Day 10 : Triggers, Pivot and Cursors in SQL
Day 14 : MySQL in Depth
Day 15 : PostgreSQL inDepth
Anyways, For Day 15 of 15 days of Advanced SQL, we will cover —
PostgreSQL inDepth
Github for Advanced SQL that you can follow —
All the projects, data structures, algorithms, system design, Data Science and ML, Data Engineering, MLOps and Deep Learning videos will be published on our youtube channel ( just launched).
Subscribe today!
System Design Case Studies — In Depth
Complete Data Structures and Algorithm Series
Github —
Some of the other best Series —
How to solve any System Design Question ( approach that you can take)?
30 days of Data Structures and Algorithms and System Design Simplified
Data Science and Machine Learning Research ( papers) Simplified **
100 days : Your Data Science and Machine Learning Degree Series with projects
Complete Data Visualization and Pre-processing Series with projects
Tech Newsletter —
If you are interested, you can join my newsletter through which I send tech interview tips, techniques, patterns, hacks — Software Development, ML, Data Science, Startups and Technology projects to more than 30K readers. You can subscribe to Tech Brew :
For Python Projects —
For complete 60 days of Data Science and ML : Day 1 — Day 60 : Quick Recap of 60 days of Data Science and ML
Follow for more updates. Stay tuned and keep coding! Some of the links are affiliates.
For other projects, tune to —
Build Machine Learning Pipelines( With Code)
Recurrent Neural Network with Keras
Clustering Geolocation Data in Python using DBSCAN and K-Means
Facial Expression Recognition using Keras
Hyperparameter Tuning with Keras Tuner
Custom Layers in Keras
