Profile.fyi

Co-Founder

Google

I worked for GCP Dataproc which provides managed spark and an entire buffet of open source big data stack. Dataproc is one of the most critical offerings available in GCP and I was super fortunate to lead the India charter.

During my stint at Google, I got to work on four really interesting technical problem statements, they were

1. building a remote shuffle service for Spark
2. building a compute resource manager for Spark
3. capture and serve data lineage for any Spark job
4. making Dataproc compliant to operate across any geography

One fun initiative I started and led was called "Dataproc Shorts" where our engineers posted a short video about the internal workings of Dataproc every week. This bite-sized content helped in building a crisp knowledge repository and gave engineers the necessary visibility across a wider org.

Staff Software Engineer

Unacademy

- Led infrastructure and SaaS cost optimization drive and saved $1M USD MoM
- Led architecting and building of Data Lake
- Led the mission-critical data migration from a managed to hosted CRM
- Built the platform to power real-time user-facing analytics using Apache Pinot

Director Of Engineering

- Laid the foundation of SRE and set up a team of 7 engineers
 - Building Data Lake to streamline central data discovery and data query
 - Conceptualized the platform to power real-time user-facing analytics
 - Designed and built in-house a secret management
 - Led efforts to set up a cross-region development environment
 - Led the development of internal tools to boost Dev Productivity
 - Led efforts to improve overall infrastructure utilization by 22%
 - Leading the efforts to make Unacademy an aspirational tech brand
 - Drove engineering campus recruitments for 2021-22
 - Drove efforts in making the infrastructure, secure, and GDPR Compliant
 - Mentorship, PIP, Devising Roadmap, Charter, and Vision of the team

Senior Engineering Manager

- Built a Gamification Engine that rewards learners for their watch time
 - Built a Home Feed engine improving platform engagement by 30%
 - Fought a data breach and drove short and long-term security initiatives
 - Initiated and drove efforts for Monolith to Micro-services migration
 - Mentoring Backend and Infrastructure Engineers

Principal Engineer

- Lead the Engineering team to model services at scale
 - Built the notifications system serving 1B notifications every day
 - Designed and developed a generic and extensible Search platform
 - Bettered the result relevance which improved CTR by ~42%
 - Designed Log archival and Visualization system
 - Built an entity and intent identification service
 - Tech: Golang, Python, Elasticsearch, NLP

Technical Architect

Amazon

- Developed a service that puts infrequently accessed data in cold tier
 - Created streamlined and seamless access for Hot and Cold Data
 - Bootstrapped a generic data pipeline platform for Amazon Retail
 - Developed a platform that enables auto-healing of services in despair
 - Tech: Java, DynamoDB, S3, EMR

Software Development Engineer II

Practo

- Developed the tool that enables incremental release of products
 - Built the product that identifies anomalies in real time
 - Built the core database service that ensures high horizontal scalability and high availability
 - Tech: Python, Flask, Java, MySQL, MongoDB, Boto3

Sr. Software Engineer Platform Engineering

- Honoured with Star Employee Award
 - Built the tool used for dummy data population across products
 - Built the tool used for managing and provisioning staging environment, instances and infrastructure
 - Zero downtime and Zero impact migration of Practo's Staging Infrastructure to a different AWS Region.
 - Tech: Puppet, Vagrant, Python, Flask, Boto3, Semantic UI

Software Engineer Platform Engineering

IIIT Hyderabad

Teaching Assistant

Saba

Head developer of Offline reviews feature and Language Check feature of Saba’s SPC product at Saba Software Pvt. Ltd.
Roles: API design and development, Performance improvement, Refactoring and design database schema and Bugs fixing.

Experience in:
- Analysis and design of databases and user interfaces and managing requirements.
- Implementing software development life cycle policies and procedures.
- Managing and supporting multiple projects.
- Highly adaptable in quickly changing technical environments with very organizational and analytical skills.
- Worked in Java EJB, Database design, Product design and Performance improvement.

Notable work done:
- Integrated ApachePOI library providing an exclusive feature - Offline Review in Saba SPC.
- Creating On-The-Fly excel to support offline review.
- Working as RnD developer to increase performance and include features in company products.

Software Associate

TCS Innovation Lab, TRDDC

- Project: Precise Detection of Uninitialised Variables using Dynamic Analysis - Extending to Aggregate and Vector Types.
 - Identified Memory Leaks and Uninitialised variables in C based program/applications using Dynamic Binary Instrumentation.
 - Received “Best Project Presentation” award for this project.

Project Trainee

I am a CS engineer passionate about building systems that scale and currently working with Google as a Staff Software Engineer with the Dataproc team and making Apache Spark faster.

Throughout my career, I have worked across a variety of domains and have built systems, services, and platforms that scale to billions; and have worked at companies like Unacademy, Amazon, Practo, and D. E. Shaw.

I hold a Master's in CS from IIIT-Hyderabad, specializing in Databases, Information Retrieval and Web Mining. I teach a cohort-based course on System Design and have taught 1000+ engineers. On the side, I am

    building a tiny in-mem database DiceDB
    running a YouTube channel with 20k subscribers
    publishing a newsletter with 21k readers
    putting out one-liners through The Smarter Chimp
    reading and implementing a bunch of research papers
    reading a few engineering books every now and then

One of the unsettling trends I observed at big tech and FAANGs is that people are more interested in critiquing ideas than in understanding them 🤦‍♂️

BigTechs have a massive promotion-driven culture. Every action one takes is kind of a tick on one of the traits for promotion. Hence the behavior stems from a desire to be 'visible' and to stand out, but it comes at the cost of belittling someone and giving up on genuine curiosity and learning.

I am not saying not to ask questions in meetings and take things at face value, but sometimes, people need to be more curious without trying to show off their superiority over others.

One simple way to do this is by having a 'no-criticism' rule until the person is ready to take questions or has covered a significant portion of the topic i.e. an 'explore first, critique later' mindset.

Again, I will reiterate that it is important to strike a balance between curiosity and critical thinking. While it's crucial to be open to new ideas, it's equally important to evaluate them rigorously before implementation. The key is to ensure that the initial phase is free from judgment.

Be curious, not judgemental.

Arpit Bhayani

Darwin Lo

I spent last week exploring the internals of DragonflyDB - a drop-in replacement of Redis, and it does a staggering 6.43 million ops/second on a single machine 🤯

DragonflyDB is a multi-threaded implementation of Redis with a bunch of insane optimizations. Some of the architectural design decisions and internals of data structure implementation just blew my mind.

To understand it better, I went through their blogs, docs, and even some fragments of source code. I compiled all my learnings in a series of videos and trust me you will absolutely love going through it.

the videos will change how you look at data structures :) subscribe to my channel, and the video drops every Friday evening.

youtube.com/c/ArpitBhayani

ps: Until then watch other System Design and Database Internals videos on my channel; they are all no-fluff.

#AsliEngineering #DatabaseInternals #SystemDesign

Taplio

Arpit Bhayani's Linkedin Analytics

Arpit Bhayani

Arpit Bhayani's Best Posts (last 30 days)

Want to drive more opportunities from LinkedIn?

Taplio

.css-1jgreyo{background-image:linear-gradient(284deg, rgba(39,43,72,1) 0%, rgba(33,47,183,1) 100%);color:transparent;-webkit-background-clip:text;background-clip:text;}Arpit Bhayani's Linkedin Analytics

Arpit Bhayani

Arpit Bhayani's Best Posts .css-qqfgvy{font-size:var(--chakra-fontSizes-md);}(last 30 days)

Want to drive more opportunities from LinkedIn?

Arpit Bhayani's Linkedin Analytics

Arpit Bhayani's Best Posts (last 30 days)