Why Does System Design Felt So Scary to Me at First

The first time in my life when I was interviewed to “design a URL shortener,” I paused for about a minute. The problems of data structures and algorithms never frightened me. I could solve linked list reversals blindfolded. I simply had no idea what kind of question this was. It did not have one correct answer, or, rather, it had too many.

Does this sound like something you can relate to? System design is usually not covered in college. However, many companies, especially those with a strong product focus and significant funding, screen candidates on system design during the entry-level interviews. I think it is fair to say that the expectations are not that high for somebody with a year of experience. They want to see if you are at least familiar with the basic concepts and can competently engage in a dialogue about system trade-offs. This cheat sheet acts as a bridge between the DSA interview and real-world system architecture. It covers ten concepts that one encounters when designing systems of various scales.

What Are Interviewers Looking For in Entry-Level Candidates?

Before diving into the concepts, one must understand what interviewers are looking for. In the entry-level system design interview, one is not expected to know database internals. There are certain cues that signal to interviewers that they got someone with potential. For example, candidates who ask clarifying questions before jumping into the design. Another positive sign is the ability to break the problem into smaller parts and explain what one is doing. Sometimes, explaining the rationale behind major and minor trade-offs is also a good sign. Candidates who mention that they would use a cache because reads are more frequent than writes is miles ahead of those who draw a diagram with a dozen components without providing any justification. Think of the concepts discussed below as keywords: you are unlikely to use all of them, but you should be able to explain any.

Ten Important Concepts Every Entry-Level Developer Should Know About

Concept 1: Scalability and Its Variants

Scalability is the property of the system to handle a more significant load without crashing. There are two primary ways to make a system scalable. These are vertical and horizontal scaling. The former entails making a single machine faster and more capable by giving it more memory and CPU power. While it is simple to implement, there is a limit to how much you can scale vertically, and the machine will hit a breaking point. Scaling out refers to adding more machines and distributing the load between them. While this approach is more involved, it allows for greater scalability and is typically the correct choice for most applications. In an interview, always note this difference and give the rationale behind the selected approach.

Concept 2: Load Balancing

After scaling out, one may wonder about the mechanism that distributes the load between servers. This is the job of a load balancer, a software component that sits in front of application servers and directs incoming requests to them. There are many ways to direct traffic, such as round robin and least connections, where requests are distributed based on the servers’ current load. Another critical function of a load balancer is to conduct health checks and hot-swap failed servers. For interviews, remember that a load balancer will increase availability and capacity.

Concept 3: Caching

A cache is a place where frequently accessed information is stored for faster access. The idea is that information which is in the cache does not need to be retrieved, computed, or transferred from one computer component to another. There is a trade-off between the freshness of data and the performance gain, which should always be articulated in an interview. A cache needs to have an expiration time for the data stored in it and a refresh mechanism. You can mention caching in any read-heavy system, and always add a sentence or two about the freshness.

Concept 4: SQL vs NoSQL Databases

Databases are the backbone of most applications, and one needs to understand what options exist. Relational databases are also called structured query languages or SQL. These databases hold data in tables and have complex querying capabilities. They are typically used to store relational data, such as a user’s purchases or their personal information, where consistency is required. The alternative to relational databases is NoSQL, which includes document, key-value, and column-oriented databases. They are used when one’s data does not fit the relational format, or one needs horizontal scaling. In an interview, never say that one option is better than the other. Always describe what kind of data the system stores and how it is accessed before giving the rationale behind the choice.

Concept 5: Database Indexing

An index is a data structure that allows faster lookup of information in a database. It is particularly useful when one needs to retrieve information based on specific fields. The analogy to real life is a table of contents at the beginning of a book: one can jump to the required chapter instead of searching the entire book. Because indexes are stored separately, this comes at a cost of extra memory. Another downside is that they slow down the write operations because the database has to update the index every time it modifies data. One should always mention indexing when designing systems if one’s design involves reading data based on particular fields.

Concept 6: Replication and Sharding

There are two ways to increase availability and fault tolerance in a system. The first is replication, which means making copies of data and storing them on different machines. This design allows presenting the same information even if one database copy becomes unavailable, thus increasing availability. It also allows sharing the read requests between the replicas, which improves performance. The second method is called sharding, and it involves distributing the data between different machines. It is similar to replication, but instead of making copies of data, shards are processed as units. Sharding allows greater horizontal scalability but complicates some types of queries. When explaining these concepts in an interview, remember that replication is about making copies of data and sharding is about distributing data.

Concept 7: The CAP Theorem

The CAP theorem is a relatively recent discovery in database design which states that in a distributed system, one cannot have all three of consistency, availability, and partition tolerance. The consistency here refers to the ability to always read the latest information, and the availability means that the system will respond to any request. Partition tolerance is the ability to maintain the system in the presence of network errors. One should always give examples of applications for which one would choose consistency or availability and explain the rationale behind the choice.

Concept 8: The Message Queue

A message queue is a mechanism of communication between different parts of a system. It allows decoupling the producer of information from the consumer and gives both sides more flexibility. The queue can be implemented as a first-in-first-out structure or have other properties. The advantages of the message queue are that it can buffer a large number of messages, thus smoothing out any sudden surges in the incoming traffic. Another benefit is that the entire system can continue functioning in case of a failure in one of the subsystems. One should always mention message queues when one’s system has to deal with slow services. Another example could be an email service which is not critical and can be retried if it fails.

Concept 9: Content Delivery Network

A content delivery network or CDN is a system of servers that caches information for faster delivery. The primary purpose of CDN is to reduce the load on the main website or application server, which makes the whole system more scalable. The way it does this is by caching static content like videos and images on CDN servers located worldwide. This design allows the users to receive the data faster because they can be anywhere on the globe. In an interview, one should always mention CDN since the added value is substantial, and the implementation is relatively simple. It can be used in almost any system that has static content.

Concept 10: Rate Limiting and API Design

Rate limiting is a technique which prevents any single user or application from monopolizing the system resources. It is typically implemented as a set of rules that specify how many requests a given user can make in a certain timeframe. The most common algorithm for rate limiting is the token bucket. One of the most common ways to implement rate limiting is to return an appropriate HTTP status code when the user exceeds the limit. Another useful technique is to return a meaningful error message in case of rate limiting. The two concepts that are closely related to rate limiting are the API design and HTTP status codes. One should always attempt to design clean and simple APIs with consistent resource naming and use HTTP status codes to indicate the outcome of an operation. It is a good idea to provide a few examples of API endpoints during the interview.

How to Approach the Interview Using These Concepts

As mentioned above, system design interviews are about trade-offs, and it is not possible to build a system that is the best in all aspects. To give an answer during an interview, one should always follow the same structure. First, clarify the requirements of the system to be designed. What is the primary function, who are the users, and how many users are there expected to be? After that, one should estimate the scale of the system roughly. How many writes and reads are expected, and what is the size of the data? The next step is to provide a high-level architecture of the system. Usually, it is sufficient to draw a client, a load balancer, some servers, a cache, and a database. Then, one should discuss the components in more detail, and explain the rationale behind the choices. Finally, it is a good idea to mention possible optimizations and system bottlenecks.

An Example Question to Walktrough

The example question that I have mentioned at the beginning of this article is probably the most common one. The task is to design a URL shortener, which is a service that takes a long URL and converts it into a shorter string. The first thing to do is to ask clarifying questions: what is the expected scale of the system, and do the shortened URLs expire? One could assume that the short URLs will be accessed more frequently than they are created, so a cache will help to reduce the database load. The system will likely need to scale out to handle a larger load, which involves a load balancer and multiple application servers. The database will store the correspondence between the short part of the URL and the full URL; it is wise to mention a hash index on the short URL for faster access. If the system grows to a considerable size, one can shard the database or replicate the database for availability. Another important design choice is rate limiting, which protects the system from being abused. A CDN is not necessary in this case, which is also a valid point to mention.

Checklist for Preparing to the Interview

One can use the above information as the basis for preparation. This is what I suggest one does to prepare for a system design interview:

Make sure that one can explain each concept concisely. One should be able to describe each term in 2-3 simple sentences.

Practice drawing a simple high-level system design from memory.

Pick a few questions and try to design a system for each.

Try to verbalize one’s thoughts as one designs the system.

Prepare a few clarifying questions to ask for any of the questions as per the interview guidelines.

Conduct a mock interview with a friend and ask for feedback.

Read some articles published by companies one wants to work for.

Mistakes to Avoid

Some common mistakes that entry-level candidates make in a system design interview are the following:

One should not jump straight to the design without asking clarifying questions.

It is important to avoid naming any particular technologies without explaining the rationale behind them.

Over-engineering is also a problem, as it shows that one does not understand the system’s needs. One should not put sharding, queues, and multiple other components when a simple solution would suffice.

Finally, one should always talk out loud, even if one is not sure about the answer.

Conclusion

In conclusion, system design interviews are not as scary as they seem. They are a stress test of one’s ability to create a viable design and communicate it clearly. One should always discuss the rationale behind each step. Furthermore, it is vital to mention one’s assumptions and talk about the different possibilities. One should aim to explain the concepts discussed here clearly and concisely. In the long run, one should design systems of various scales and increase one’s quality of system design with each iteration.