ByteByteGo

Co-authored 4 books:
     - System Design Interview - Volume 1
     - System Design Interview - Volume 2
     - Coding Interview Patterns
     - Machine Learning System Design Interview

- Write weekly ByteByteGo newsletter on system design topics to 1,000,000+ subscribers.
- Grow the ByteByteGo YouTube channel to 1,000,000+ subscribers.

Author & Co-Founder

Twitter

Software Engineer

Apple

Zynga

Oracle

Software Developer

Modeln N

Associate Member of Technical Staff

Follow me for system design & book-writing tips.

Alex Xu is a software engineer and author. His book System Design Interview - An Insider’s Guide is one of Amazon's best-selling books in the “Distributed Systems & Computing” category.  As of December 2020, his book is being translated into 7 different languages (Traditional Chinese, Russian, Japanese, Spanish, Polish, Simplified Chinese, and Korean).

--------------
Newsletter: https://blog.bytebytego.com
Twitter: https://twitter.com/alexxubyte
YouTube: https://www.youtube.com/c/ByteByteGo

Visualizing a SQL query

SQL statements are executed by the database system in several steps, including:
- Parsing the SQL statement and checking its validity
- Transforming the SQL into an internal representation, such as relational algebra
- Optimizing the internal representation and creating an execution plan that utilizes index information
- Executing the plan and returning the results

-- 
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social

#systemdesign #coding #interviewtips 
.

Alex Xu

Follow me for system design & book-writing tips.

Alex Xu is a software engineer and author. His book System Design Interview - An Insider’s Guide is one of Amazon's best-selling books in the “Distributed Systems & Computing” category.  As of December 2020, his book is being translated into 7 different languages (Traditional Chinese, Russian, Japanese, Spanish, Polish, Simplified Chinese, and Korean).

--------------
Newsletter: https://blog.bytebytego.com
Twitter: https://twitter.com/alexxubyte
YouTube: https://www.youtube.com/c/ByteByteGo
Talent Collective: https://bytebytego.pallet.com/jobs

Author of 4 Bestselling Books | Co-Founder of ByteByteGo

Rakuram E

Explaining JSON Web Token (JWT) with simple terms.

Imagine you have a special box called a JWT. Inside this box, there are three parts: a header, a payload, and a signature.

The header is like the label on the outside of the box. It tells us what type of box it is and how it's secured. It's usually written in a format called JSON, which is just a way to organize information using curly braces { } and colons : .

The payload is like the actual message or information you want to send. It could be your name, age, or any other data you want to share. It's also written in JSON format, so it's easy to understand and work with.

Now, the signature is what makes the JWT secure. It's like a special seal that only the sender knows how to create. The signature is created using a secret code, kind of like a password. This signature ensures that nobody can tamper with the contents of the JWT without the sender knowing about it.

When you want to send the JWT to a server, you put the header, payload, and signature inside the box. Then you send it over to the server. The server can easily read the header and payload to understand who you are and what you want to do.

Over to you: When should we use JWT for authentication? What are some other authentication methods?

-- 
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social

#systemdesign #coding #interviewtips 
.

The Open Source AI Stack

You don’t need to spend a fortune to build an AI application. The best AI developer tools are open-source, and an excellent ecosystem is evolving that can make AI accessible to everyone.

The key components of this open-source AI stack are as follows:

1 - Frontend
To build beautiful AI UIs, frameworks like NextJS and Streamlit are extremely useful. Also, Vercel can help with deployment.

2 - Embeddings and RAG libraries
Embedding models and RAG libraries like Nomic, JinaAI, Cognito, and LLMAware help developers build accurate search and RAG features.

3 - Backend and Model Access
For backend development, developers can rely on frameworks like FastAPI, Langchain, and Netflix Metaflow. Options like Ollama and Huggingface are available for model access.

4 - Data and Retrieval
For data storage and retrieval, several options like Postgres, Milvus, Weaviate, PGVector, and FAISS are available.

5 - Large-Language Models
Based on performance benchmarks, open-source models like Llama, Mistral, Qwen, Phi, and Gemma are great alternatives to proprietary LLMs like GPT and Claude.

Over to you: Which other tool will you add to the Open Source AI Stack?

– 
Subscribe to our newsletter to download the 𝐡𝐢𝐠𝐡-𝐫𝐞𝐬𝐨𝐥𝐮𝐭𝐢𝐨𝐧 𝐜𝐡𝐞𝐚𝐭 𝐬𝐡𝐞𝐞𝐭. After signing up, find the download link on the success page: https://bit.ly/3tiMC1B

#systemdesign #coding #interviewtips 
.

Big O Notation 101: The Secret to Writing Efficient Algorithms 
 
From simple array operations to complex sorting algorithms, understanding the Big O Notation is critical for building high-performance software solutions. 
 
1 - O(1) 
This is the constant time notation. The runtime remains steady regardless of input size. For example, accessing an element in an array by index and inserting/deleting an element in a hash table. 
 
2 - O(n) 
Linear time notation. The runtime grows in direct proportion to the input size. For example, finding the max or min element in an unsorted array. 
 
3 - O(log n) 
Logarithmic time notation. The runtime increases slowly as the input grows. For example, a binary search on a sorted array and operations on balanced binary search trees. 
 
4 - O(n^2) 
Quadratic time notation. The runtime grows exponentially with input size. For example, simple sorting algorithms like bubble sort, insertion sort, and selection sort. 
 
5 - O(n^3) 
Cubic time notation. The runtime escalates rapidly as the input size increases. For example, multiplying two dense matrices using the naive algorithm. 
 
6 - O(n logn) 
Linearithmic time notation. This is a blend of linear and logarithmic growth. For example, efficient sorting algorithms like merge sort, quick sort, and heap sort 
 
7 - O(2^n) 
Exponential time notation. The runtime doubles with each new input element. For example, recursive algorithms solve problems by dividing them into multiple subproblems. 
 
8 - O(n!) 
Factorial time notation. Runtime skyrockets with input size. For example, permutation-generation problems. 
 
9 - O(sqrt(n)) 
Square root time notation. Runtime increases relative to the input’s square root. For example, searching within a range such as the Sieve of Eratosthenes for finding all primes up to n. 
 
Over to you: What else will you add to better understand the Big O Notation? 

– 
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social

#systemdesign #coding #interviewtips 
.

12 MCP Servers You Can Use in 2025

MCP (Model Context Protocol) is an open standard that simplifies how AI models, particularly LLMs, interact with external data sources, tools, and services. An MCP server acts as a bridge between these AI models and external tools. Here are the top MCP servers:

1 - File System MCP Server
Allows the LLM to directly access the local file system to read, write, and create directories.

2 - GitHub MCP Server
Connects Claude to GitHub repos and allows file updates, code searching.

3 - Slack MCP Server
MCP Server for Slack API, enabling Claude to interact with Slack workspaces.

4 - Google Maps MCP Server
MCP Server for Google Maps API.

5 - Docker MCP Server
Integrate with Docker to manage containers, images, volumes, and networks.

6 - Brave MCP Server
Web and local search using Brave’s Search API.

7 - PostgreSQL MCP Server
An MCP server that enables LLM to inspect database schemas and execute read-only queries.

8 - Google Drive MCP Server
An MCP server that integrates with Google Drive to allow reading and searching over files.

9 - Redis MCP Server
MCP Server that provides access to Redis databases.

10 - Notion MCP Server
This project implements an MCP server for the Notion API.

11 - Stripe MCP Server
MCP Server to interact with the Stripe API.

12 - Perplexity MCP Server
An MCP Server that connects to Perplexity’s Sonar API for real-time search.

Over to you: Which other MCP Server will you add to the list?

– 
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social

#systemdesign #coding #interviewtips 
.

No alternative text description for this image

Oauth 2.0 Explained With Simple Terms. 
. 
. 
OAuth 2.0 is a powerful and secure framework that allows different applications to securely interact with each other on behalf of users without sharing sensitive credentials. 
 
The entities involved in OAuth are the User, the Server, and the Identity Provider (IDP). 
 
What Can an OAuth Token Do? 
 
When you use OAuth, you get an OAuth token that represents your identity and permissions. This token can do a few important things: 
 
Single Sign-On (SSO): With an OAuth token, you can log into multiple services or apps using just one login, making life easier and safer. 
 
Authorization Across Systems: The OAuth token allows you to share your authorization or access rights across various systems, so you don't have to log in separately everywhere. 
 
Accessing User Profile: Apps with an OAuth token can access certain parts of your user profile that you allow, but they won't see everything. 
 
Remember, OAuth 2.0 is all about keeping you and your data safe while making your online experiences seamless and hassle-free across different applications and services. 
 
Over to you: Imagine you have a magical power to grant one wish to OAuth 2.0. What would that be? Maybe your suggestions actually lead to OAuth 3. 

-- 
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social

#systemdesign #coding #interviewtips 
.

How to Learn Cloud Computing?

Cloud computing is a vast field with an ever-growing footprint. It can often get tricky for a new developer to understand where to start. Here’s a learning map:

1 - Cloud Computing Basics
This includes topics such as “what is cloud computing,” its benefits, cloud models (public, private, hybrid, and multi), and a comparison of cloud vs. on-premise.

2 - Cloud Service Models
Learn about cloud service models such as IaaS, PaaS, and SaaS.

3 - Cloud Providers
Explore the various popular cloud platforms such as AWS, Azure, GCP, Oracle Cloud, IBM Cloud, etc. Also, learn how to choose a cloud provider.

4 - Key Cloud Services
Learn the key cloud services related to Compute (EC2, Azure VM, Docker, Kubernetes, Lambda, etc), Storage (EBS, Azure Disk, S3, Azure Blob, EFS, etc.), and Networking (VPC, ELB, Azure LB, Cloudfront, and Azure CDN).

5 - Security & Compliance
Learn about the critical security and compliance points related to identity, access management, encryption, data security, DDoS protection, and WAF.

6 - Cloud DevOps & Automation
Learn Cloud DevOps and automation in specific areas such as CI/CD, IaC, and Monitoring.

Over to you: What else will you add to the list for learning cloud computing?

-- 
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social

#systemdesign #coding #interviewtips 
.

A Cheatsheet on REST API Design Best Practices.

Well-designed APIs behave consistently, fair predictably, and grow without friction. Some best practices to keep in mind are as follows:

1 - Resource-oriented paths and proper use of HTTP verbs help APIs align with standard tools.
2 - Use a proper API versioning approach.
3 - Use standard error codes while generating API responses.
4 - APIs should be idempotent. They ensure safe retries by making repeated requests to produce the same result, especially for POST operations.
5 - Idempotency keys allow clients to safely deduplicate operations with side effects.
6 - APIs should support pagination to prevent performance bottlenecks and payload bloat. Some common pagination strategies are offset-based, cursor-based, and keyset-based.
7 - API security is mandatory for well-designed APIs. Use proper authentication and authorization with APIs using API Keys, JWTs, OAuth2, and other mechanisms. HTTPS is also a must-have for APIs running in production.

Over to you: Which other best practices do you follow while designing APIs?

-- 
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social

#systemdesign #coding #interviewtips 
.

Sergey Sundukovskiy, Ph.D.

The 4 Types of SQL Joins

Edit: The right join output should be 4. Updated diagram in the comment

SQL joins combine rows from two or more tables based on a related column. Here are the different types of joins you can use:

1 - Inner Join
Returns only the matching rows between both tables. It keeps common data only.

2 - Left Join
Returns all rows from the left table and matching rows from the right table. If a row in the left table doesn’t have a match in the right table, the right table’s columns will contain NULL values in that row.

3 - Right Join
Returns all rows from the right table and matching rows from the left table. If no matching record exists in the left table for a record in the right table, the columns from the left table in the result will contain NULL values. 

4 - FULL OUTER JOIN
Returns all rows from both tables, filling in NULL for missing matches. 

Over to you: Which SQL Join have you used the most?

-- 
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social

#systemdesign #coding #interviewtips 
.

The Fundamental Pillars of Object-Oriented Programming

Abstraction, Encapsulation, Inheritance, and Polymorphism are the four pillars of object-oriented programming. What do they mean?

1 - Abstraction
This is the process of hiding implementation details and showing only the essential features of an object. For example, a Vehicle class with an abstract stop method.

2 - Encapsulation
It involves wrapping data (fields) and methods in a single unit (class) and restricting direct access using access modifiers. For example, private fields with public getters and setters.

3 - Inheritance
The process of creating a new class (child) that inherits attributes and methods from an existing class (parent), thereby promoting code reuse. For example, a Car class inherits from a Vehicle class.

4 - Polymorphism
It allows methods to perform differently based on the object they are invoked on. When two types share an inheritance chain, they can be used interchangeably with no errors.

Over to you: Do you use object-oriented programming?

--
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social
 
#systemdesign #coding #interviewtips 
.

9 Clean Code Principles To Keep In Mind

1 - Meaningful Names: Name variables and functions to reveal their purpose, not just their value.
2 - One Function, One Responsibility: Functions should do one thing. 
3 - Avoid Magic Numbers: Replace hard-code values with named constants to give them meaning.
4 - Use Descriptive Booleans: Boolean names should state a condition, not just its value.
5 - Keep Code DRY: Duplicate code means duplicate bugs. Try and reuse logic where it makes sense.
6 - Avoid Deep Nesting: Flatten your code flow to improve clarity and reduce cognitive load.
7 - Comment Why, Not What: Explain the intention behind your code, not the obvious mechanics.
8 - Limit Function Arguments: Too many parameters confuse. Group related data into objects.
9 - Code Should Be Self-Explanatory: Well-written code needs fewer comments because it reads like a story.

Over to you: Which other clean code principle will you add to the list?

-- 
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social

#systemdesign #coding #interviewtips 
.

Top 6 Tools to Turn Code into Beautiful Diagrams 

- Diagrams 
- Go Diagrams 
- Mermaid 
- PlantUML 
- ASCII diagrams 
- Markmap 
 
Over to you: Did we miss anything? What's your favorite?

--
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social
 
#systemdesign #coding #interviewtips 
.

Feng Yuan

Linux Boot Process Illustrated 
 
The diagram below shows the steps. 
 
Step 1 - When we turn on the power, BIOS (Basic Input/Output System) or UEFI (Unified Extensible Firmware Interface) firmware is loaded from non-volatile memory, and executes POST (Power On Self Test). 
 
Step 2 - BIOS/UEFI detects the devices connected to the system, including CPU, RAM, and storage. 
 
Step 3 - Choose a booting device to boot the OS from. This can be the hard drive, the network server, or CD ROM. 
 
Step 4 - BIOS/UEFI runs the boot loader (GRUB), which provides a menu to choose the OS or the kernel functions. 
 
Step 5 - After the kernel is ready, we now switch to the user space. The kernel starts up systemd as the first user-space process, which manages the processes and services, probes all remaining hardware, mounts filesystems, and runs a desktop environment. 
 
Step 6 - systemd activates the default. target unit by default when the system boots. Other analysis units are executed as well. 
 
Step 7 - The system runs a set of startup scripts and configure the environment. 
 
Step 8 - The users are presented with a login window. The system is now ready. 

-- 
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social

#systemdesign #coding #interviewtips 
.

The Large-Language Model Glossary

This glossary can be divided into high-level categories

1 - Models: Includes the types of models such as Foundation, Instruction-Tuned, Multi-modal, Reasoning, and Small Language Model.
2 - Training LLM: Training begins with pretraining RLHF, DPO, and Synthetic Data. Fine-Tuning adds control with datasets, checkpoints, LoRA/QLoRA, guardrails, and parameter tunings.
3 - Prompts: Prompts drive how models respond using User/System Prompts, Chain of Thought, of Few/Zero-Shot learning. Prompt Tuning and large Context Windows help shape more precise, multi-turn conversations.
4 - Inference: This is how models generate responses. Key factors include Temperature, Max Tokens, Seed, and Latency. Hallucination is a common issue here, where the model makes things up that sound real.
5 - Retrieval-Augmented Generation: RAG improves accuracy by fetching real-world data. It uses Retrieval, Semantic Search, Chunks, Embeddings, and VectorDBs. Reranking and Indexing ensure the best answers are surfaced, not just the most likely ones.

Over to you: What else will you add to the LLM glossary?

-- 
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social

#systemdesign #coding #interviewtips 
.

A handy cheat sheet for the most popular cloud services. 

Link to the high resolution at the end.
 
What’s included? 
- AWS, Azure, Google Cloud, Oracle Cloud
- Cloud servers 
- Databases 
- Message queues and streaming platforms 
- Load balancing, DNS routing software 
- Security 
- Monitoring 
 
Over to you - which company is the best at naming things? 

– 
Subscribe to our newsletter to download the high-resolution cheat sheet. After signing up, find the download link on the success page: https://bit.ly/3tiMC1B

#systemdesign #coding #interviewtips 
.

What is SSO (Single Sign-On)?
.
.
Basically, Single Sign-On (SSO) is an authentication scheme. It allows a user to log in to different systems using a single ID.

The diagram below illustrates how SSO works.

Step 1: A user visits Gmail, or any email service. Gmail finds the user is not logged in and so redirects them to the SSO authentication server, which also finds the user is not logged in. As a result, the user is redirected to the SSO login page, where they enter their login credentials.

Steps 2-3: The SSO authentication server validates the credentials, creates the global session for the user, and creates a token.

Steps 4-7: Gmail validates the token in the SSO authentication server. The authentication server registers the Gmail system, and returns “valid.” Gmail returns the protected resource to the user.

Step 8: From Gmail, the user navigates to another Google-owned website, for example, YouTube. 

Steps 9-10: YouTube finds the user is not logged in, and then requests authentication. The SSO authentication server finds the user is already logged in and returns the token.

Step 11-14: YouTube validates the token in the SSO authentication server. The authentication server registers the YouTube system, and returns “valid.” YouTube returns the protected resource to the user.

The process is complete and the user gets back access to their account.

Over to you: 
Question 1: have you implemented SSO in your projects? What is the most difficult part?
Question 2: what’s your favorite sign-in method and why?

-- 
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social

#systemdesign #coding #interviewtips 
.

The System Design Topic Map

Effective system design is a game of trade-offs and requires a broad knowledge base to make the best decisions. This topic map categorizes the essential system design topics based on categories.

1 - Application Layer: It consists of the core concepts such as availability, scalability, reliability, and other NFRs. Also covers design and architectural topics such as OOP, DDD, Microservices, Clean Architecture, Modular Monoliths, and so on.

2 - Network & Communication: It covers communication protocols, service integration, messaging, real-time communication, and event-driven architecture.

3 - Data Layer: It covers the basics of database systems (schema design, indexing, SQL vs NoSQL, transactions, etc), the various types of databases, and the nuances of distributed databases (replication, sharding, leader election, etc.)

4 - Scalability & Reliability: This covers scalability strategies (horizontal, stateless, caching, partitioning, etc) and reliability strategies like load balancing, rate limiting, and so on.

5 - Security & Observability: It covers authentication and authorization techniques (OAuth 2, JWT, PASETO, Sessions, Cookies, RBAC, etc.) and security threats. The observability area deals with topics like monitoring, tracing, and logging.

6 - Infrastructure & Deployments: Deals with CI/CD pipelines, containerization and orchestration, serverless architecture, IaC, and disaster recovery techniques.

Over to you: What else will you add to the list?

--
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social
 
#systemdesign #coding #interviewtips 
.

Top 30 AWS Services That Are Commonly Used. 

We group them by category and understand what they do.

Compute Services
1 - Amazon EC2: Virtual servers in the cloud
2 - AWS Lambda: Serverless functions for event-driven workloads
3 - Amazon ECS: Managed container orchestration
4 - Amazon EKS: Kubernetes cluster management service
5 - AWS Fargate: Serverless compute for containers

Storage Services
6 - Amazon S3: Scalable secure object storage
7 - Amazon EBS: Block storage for EC2 instances
8 - Amazon FSx: Fully managed file storage
9 - AWS Backup: Centralized backup automation
10 - Amazon Glacier: Archival cold storage for backups

Database Services
11 - Amazon RDS: Managed relational database service
12 - Amazon DynamoDB: NoSQL database with low latency
13 - Amazon Aurora: High-performance cloud-native database
14 - Amazon Redshift: Scalable data warehousing solution
15 - Amazon Elasticache: In-memory caching with Redis/Memcached
16 - Amazon DocumentDB: NoSQL document database (MongoDB-compatible)
17 - Amazon Keyspaces: Managed Cassandra database service

Networking & Security
18 - Amazon VPC: Secure cloud networking
19 - AWS CloudFront: Content Delivery Network
20 - AWS Route53: Scalable domain name system (DNS)
21 - AWS WAF: Protects web applications from attacks
22 - AWS Shield: DDoS protection for AWS workloads

AI & Machine Learning
23 - Amazon SageMaker: Build, train, and deploy ML models
24 - AWS Rekognition: Image and video analysis with AI
25 - AWS Textract: Extracts text from scanned documents
26 - Amazon Comprehend: AI-driven natural language processing

Monitoring & DevOps
27 - Amazon CloudWatch: AWS performance monitoring and alerts
28 - AWS X-Ray: Distributed tracing for applications
29 - AWS CodePipeline: CI/CD automation for deployments
30 - AWS CloudFormation - Infrastructure as Code (IaC)

Over to you: Which other AWS service will you add to the list?

-- 
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social

#systemdesign #coding #interviewtips 
.

Deploying or upgrading services is risky. In this post, we explore risk mitigation strategies.

The diagram below illustrates the common ones.

Multi-Service Deployment
In this model, we deploy new changes to multiple services simultaneously. This approach is easy to implement. But since all the services are upgraded at the same time, it is hard to manage and test dependencies. It’s also hard to rollback safely.

Blue-Green Deployment
With blue-green deployment, we have two identical environments: one is staging (blue) and the other is production (green). The staging environment is one version ahead of production. Once testing is done in the staging environment, user traffic is switched to the staging environment, and the staging becomes the production. This deployment strategy is simple to perform rollback, but having two identical production quality environments could be expensive.

Canary Deployment
A canary deployment upgrades services gradually, each time to a subset of users. It is cheaper than blue-green deployment and easy to perform rollback. However, since there is no staging environment, we have to test on production. This process is more complicated because we need to monitor the canary while gradually migrating more and more users away from the old version.

A/B Test
In the A/B test, different versions of services run in production simultaneously. Each version runs an “experiment” for a subset of users. A/B test is a cheap method to test new features in production. We need to control the deployment process in case some features are pushed to users by accident.

Over to you - Which deployment strategy have you used? Did you witness any deployment-related outages in production and why did they happen?

--
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social
 
#systemdesign #coding #interviewtips 
.

How Transformers Architecture Works?

Transformers Architecture has become the foundation of some of the most popular LLMs including GPT, Gemini, Claude, DeepSeek, and Llama.

Here’s how it works:

1 - A typical transformer-based model has two main parts: encoder and decoder. The encoder reads and understands the input. The decoder uses this understanding to generate the correct output.
2 - In the first step (Input Embedding), each word is converted into a number (vector) representing its meaning.
3 - Next, a pattern called Positional Encoding tells the model where each word is in the sentence. This is because the word order matters in a sentence. For example “the cat ate the fish” is different from “the fish ate the cat”.
4 - Next is the Multi-Head Attention, which is the brain of the encoder. It allows the model to look at all words at once and determine which words are related. In the Add & Normalize phase, the model adds what it learned from attention back into the sentence.
5 - The Feed Forward process adds extra depth to the understanding. The overall process is repeated multiple times so that the model can deeply understand the sentence.
6 - After the encoder finishes, the decoder kicks into action. The output embedding converts each word in the expected output into numbers. To understand where each word should go, we add Positional Encoding.
7 - The Masked Multi-Head Attention hides the future words so the model predicts only one word at a time.
8 - The Multi-Head Attention phase aligns the right parts of the input with the right parts of the output. The decoder looks at both the input sentence and the words it has generated so far.
9 - The Feed Forward applies more processing to make the final word choice better. The process is repeated several times to refine the results.
10 - Once the decoder has predicted numbers for each word, it passes them through a Linear Layer to prepare for output. This layer maps the decoder’s output to a large set of possible words.
11 - After the Linear Layer generates scores for each word, the Softmax layer converts those scores into probabilities. The word with the highest probability is chosen as the next word. 
12 - Finally, a human-readable sentence is generated.

Over to you: What else will you add to understand the Transformer Architecture?

--
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social
 
#systemdesign #coding #interviewtips 
.

Taplio

Alex Xu's Linkedin Analytics

Who is engaging with Alex

's Best Posts (last 30 days)

Want to drive more opportunities from LinkedIn?

Taplio

.css-1jgreyo{background-image:linear-gradient(284deg, rgba(39,43,72,1) 0%, rgba(33,47,183,1) 100%);color:transparent;-webkit-background-clip:text;background-clip:text;}Alex Xu's Linkedin Analytics

Who is engaging with Alex

's Best Posts .css-qqfgvy{font-size:var(--chakra-fontSizes-md);}(last 30 days)

Want to drive more opportunities from LinkedIn?

Alex Xu's Linkedin Analytics

's Best Posts (last 30 days)