Microsoft

SDE-1

Infosys

Specialist Progammer

Digital Specialist Engineer

SDE @Microsoft | Ex-Infosys | 133K+ on LinkedIn | 2000+ rated on LeetCode | Code, Content and Memes

I'm a Software Development Engineer at Microsoft and an ex-Infosys employee. I have over 146k followers on LinkedIn and I'm passionate about coding, creating content, and sharing memes. I'm passionate about helping others learn and grow in their professional career.


Codechef 5 star ( 2003 ) | Hackerrank 6 star | HackerEarth 1918 | Leetcode 2112 | CodeForces specialist | Highest rated coder in college | SDE at  Microsoft | Ex-Infosys | Techgig-Code Gladiator finalist 2019 , 2020 ,2021, 2022, 2023,2024

Everyone suddenly works at Cloudflare now after the 18 Nov outage (me included 😄)

On 18 Nov 2025, Cloudflare faced a global outage - and the interesting part is:
 👉 it wasn’t a cyber attack
 👉 it wasn’t a DDoS
 👉 it wasn’t hardware failure

It happened because of a small configuration change that triggered a chain reaction.

𝗟𝗲𝘁 𝗺𝗲 𝘁𝗿𝘆 𝘁𝗼 𝗲𝘅𝗽𝗹𝗮𝗶𝗻 𝘁𝗵𝗲 𝗥𝗼𝗼𝘁 𝗖𝗮𝘂𝘀𝗲 𝗶𝗻 𝗺𝗼𝘀𝘁 𝘀𝗶𝗺𝗽𝗹𝗲𝗿 𝘄𝗮𝘆 -

Cloudflare uses a ClickHouse database with 𝘁𝘄𝗼 schemas:
𝗱𝗲𝗳𝗮𝘂𝗹𝘁 → main production data
𝗿𝟬 → internal metadata / backup tables

Before the change, the system could only read from the default schema.
So a query like 👇

𝙎𝙀𝙇𝙀𝘾𝙏 * 

𝙁𝙍𝙊𝙈 𝙨𝙮𝙨𝙩𝙚𝙢.𝙘𝙤𝙡𝙪𝙢𝙣𝙨 

𝙒𝙃𝙀𝙍𝙀 𝙩𝙖𝙗𝙡𝙚 = '𝙝𝙩𝙩𝙥_𝙧𝙚𝙦𝙪𝙚𝙨𝙩𝙨_𝙛𝙚𝙖𝙩𝙪𝙧𝙚𝙨';

returned only this:

𝗦𝗰𝗵𝗲𝗺𝗮               𝗖𝗼𝗹𝘂𝗺𝗻
-----------------------------
default                ip_address
default                user_agent
default                country

🔧 The Change
Engineers updated permissions:

𝐆𝐑𝐀𝐍𝐓 𝐒𝐄𝐋𝐄𝐂𝐓 𝐎𝐍 𝐫𝟎.* 𝐓𝐎 𝐛𝐨𝐭_𝐟𝐞𝐚𝐭𝐮𝐫𝐞_𝐠𝐞𝐧𝐞𝐫𝐚𝐭𝐨𝐫;

Meaning:
➡️ “Now the system can also access r0.”

This was likely intentional - maybe for migration or diagnostics.

⚠️ The Hidden Bug

The query didn’t filter by schema.

𝐓𝐡𝐞 𝐚𝐜𝐭𝐮𝐚𝐥 𝐪𝐮𝐞𝐫𝐲 𝐰𝐚𝐬:
-------------------------

SELECT name 
FROM system.columns
WHERE table = 'http_requests_features'

𝐈𝐭 𝐬𝐡𝐨𝐮𝐥𝐝 𝐡𝐚𝐯𝐞 𝐛𝐞𝐞𝐧:
--------------------------

SELECT name 
FROM system.columns
WHERE table = 'http_requests_features'
𝐀𝐍𝐃 𝐝𝐚𝐭𝐚𝐛𝐚𝐬𝐞 = '𝐝𝐞𝐟𝐚𝐮𝐥𝐭';

So after permission change, the system fetched both schemas - resulting in duplicate rows.

📈 After the Change
Row count went from:

3 rows → 6 rows
Example:

ip_address(from default)
user_agent(from default)
country(from default)
𝐢𝐩_𝐚𝐝𝐝𝐫𝐞𝐬𝐬  (𝐝𝐮𝐩𝐥𝐢𝐜𝐚𝐭𝐞, from r0)
𝐮𝐬𝐞𝐫_𝐚𝐠𝐞𝐧𝐭  (𝐝𝐮𝐩𝐥𝐢𝐜𝐚𝐭𝐞, from r0)
𝐜𝐨𝐮𝐧𝐭𝐫𝐲   (𝐝𝐮𝐩𝐥𝐢𝐜𝐚𝐭𝐞, from r0)

💥 Why Did It Break?
Cloudflare's proxy engine (FL2) was designed with a limit:
➡️ Max ~200 features allowed

After duplication, the feature file exceeded that limit.
💾 Memory overflow
😰 Panic
💥 Crash
🌍 Global outage

🧩 𝐊𝐞𝐲 𝐓𝐚𝐤𝐞𝐚𝐰𝐚𝐲𝐬
✔️ Small assumptions can break large systems
 ✔️ Configuration changes must be validated
 ✔️ Queries must define clear scope (schema, filters, limits)
 ✔️ Reliability is not just about smart design - it's about preventing unexpected interactions

Follow Chandan for more 💥💥

#design #tech #dsa #coding #cloudflare #maang #google #microsoft

Chandan Agrawal

Both 𝐂𝐡𝐚𝐭𝐆𝐏𝐓 & Twitter Are Down 😞

Proof that the entire internet is now run by a handful of giant vendors: both services threw up a very familiar "𝐂𝐥𝐨𝐮𝐝𝐟𝐥𝐚𝐫𝐞 𝐄𝐫𝐫𝐨𝐫" at the same time.

Cloudflare isn't just a challenge screen, it's the digital backbone for most of the web's biggest players.

𝐖𝐡𝐚𝐭 𝐢𝐭 𝐝𝐨𝐞𝐬: Acts as a global bouncer (security/DDoS protection) and a ridiculously fast delivery truck (Content Delivery Network or CDN).

When one piece of the infrastructure fails, the entire digital saas bahu drama (Twitter/X) and the new AI brain trust (ChatGPT) can go offline instantly. The internet is faster, but perhaps also more fragile.

Time to grab some coffee and wait for the digital delivery guys to get the network back up.

#tech #coding #cloudflare #outage #twitter #meme

I'm a Software Development Engineer at Microsoft and an ex-Infosys employee. I have over 85k+ followers on LinkedIn and I'm passionate about coding, creating content, and sharing memes. I'm passionate about helping others learn and grow in their professional career.


Codechef 5 star ( 2003 ) | Hackerrank 6 star | HackerEarth 1918 | Leetcode 2012 | CodeForces specialist | Highest rated coder in college | SDE at  Microsoft | Ex-Infosys | Techgig-Code Gladiator finalist 2019 , 2020 ,2021

taplio

Chandan Agrawal's Linkedin Analytics

Chandan Agrawal

Performance ScoreChandan Agrawal's LinkedIn Performance

Chandan Agrawal's Best Posts (last 30 days)

Ready to grow your LinkedIn presence?

taplio

.css-6ytdur{background-image:linear-gradient(284deg, rgba(39, 43, 72, 1) 0%, rgba(33, 47, 183, 1) 100%);color:transparent;-webkit-background-clip:text;background-clip:text;}Chandan Agrawal's Linkedin Analytics

Chandan Agrawal

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Chandan Agrawal's LinkedIn Performance

Chandan Agrawal's Best Posts .css-1emoo84{font-size:var(--chakra-fontSizes-md);font-weight:400;color:var(--chakra-colors-gray-600);}(last 30 days)

Ready to grow your LinkedIn presence?

Chandan Agrawal's Linkedin Analytics

Performance ScoreChandan Agrawal's LinkedIn Performance

Chandan Agrawal's Best Posts (last 30 days)