Profile.fyi

Co-Founder

Google

I worked for GCP Dataproc which provides managed spark and an entire buffet of open source big data stack. Dataproc is one of the most critical offerings available in GCP and I was super fortunate to lead the India charter.

During my stint at Google, I got to work on four really interesting technical problem statements, they were

1. building a remote shuffle service for Spark
2. building a compute resource manager for Spark
3. capture and serve data lineage for any Spark job
4. making Dataproc compliant to operate across any geography

One fun initiative I started and led was called "Dataproc Shorts" where our engineers posted a short video about the internal workings of Dataproc every week. This bite-sized content helped in building a crisp knowledge repository and gave engineers the necessary visibility across a wider org.

Staff Software Engineer

Unacademy

- Led infrastructure and SaaS cost optimization drive and saved $1M USD MoM
- Led architecting and building of Data Lake
- Led the mission-critical data migration from a managed to hosted CRM
- Built the platform to power real-time user-facing analytics using Apache Pinot

Director Of Engineering

- Laid the foundation of SRE and set up a team of 7 engineers
 - Building Data Lake to streamline central data discovery and data query
 - Conceptualized the platform to power real-time user-facing analytics
 - Designed and built in-house a secret management
 - Led efforts to set up a cross-region development environment
 - Led the development of internal tools to boost Dev Productivity
 - Led efforts to improve overall infrastructure utilization by 22%
 - Leading the efforts to make Unacademy an aspirational tech brand
 - Drove engineering campus recruitments for 2021-22
 - Drove efforts in making the infrastructure, secure, and GDPR Compliant
 - Mentorship, PIP, Devising Roadmap, Charter, and Vision of the team

Senior Engineering Manager

- Built a Gamification Engine that rewards learners for their watch time
 - Built a Home Feed engine improving platform engagement by 30%
 - Fought a data breach and drove short and long-term security initiatives
 - Initiated and drove efforts for Monolith to Micro-services migration
 - Mentoring Backend and Infrastructure Engineers

Principal Engineer

- Lead the Engineering team to model services at scale
 - Built the notifications system serving 1B notifications every day
 - Designed and developed a generic and extensible Search platform
 - Bettered the result relevance which improved CTR by ~42%
 - Designed Log archival and Visualization system
 - Built an entity and intent identification service
 - Tech: Golang, Python, Elasticsearch, NLP

Technical Architect

Amazon

- Developed a service that puts infrequently accessed data in cold tier
 - Created streamlined and seamless access for Hot and Cold Data
 - Bootstrapped a generic data pipeline platform for Amazon Retail
 - Developed a platform that enables auto-healing of services in despair
 - Tech: Java, DynamoDB, S3, EMR

Software Development Engineer II

Practo

- Developed the tool that enables incremental release of products
 - Built the product that identifies anomalies in real time
 - Built the core database service that ensures high horizontal scalability and high availability
 - Tech: Python, Flask, Java, MySQL, MongoDB, Boto3

Sr. Software Engineer Platform Engineering

- Honoured with Star Employee Award
 - Built the tool used for dummy data population across products
 - Built the tool used for managing and provisioning staging environment, instances and infrastructure
 - Zero downtime and Zero impact migration of Practo's Staging Infrastructure to a different AWS Region.
 - Tech: Puppet, Vagrant, Python, Flask, Boto3, Semantic UI

Software Engineer Platform Engineering

IIIT Hyderabad

Teaching Assistant

Saba

Head developer of Offline reviews feature and Language Check feature of Saba’s SPC product at Saba Software Pvt. Ltd.
Roles: API design and development, Performance improvement, Refactoring and design database schema and Bugs fixing.

Experience in:
- Analysis and design of databases and user interfaces and managing requirements.
- Implementing software development life cycle policies and procedures.
- Managing and supporting multiple projects.
- Highly adaptable in quickly changing technical environments with very organizational and analytical skills.
- Worked in Java EJB, Database design, Product design and Performance improvement.

Notable work done:
- Integrated ApachePOI library providing an exclusive feature - Offline Review in Saba SPC.
- Creating On-The-Fly excel to support offline review.
- Working as RnD developer to increase performance and include features in company products.

Software Associate

TCS Innovation Lab, TRDDC

- Project: Precise Detection of Uninitialised Variables using Dynamic Analysis - Extending to Aggregate and Vector Types.
 - Identified Memory Leaks and Uninitialised variables in C based program/applications using Dynamic Binary Instrumentation.
 - Received “Best Project Presentation” award for this project.

Project Trainee

I am a CS engineer passionate about building systems that scale and currently working with Google as a Staff Software Engineer with the Dataproc team and making Apache Spark faster.

Throughout my career, I have worked across a variety of domains and have built systems, services, and platforms that scale to billions; and have worked at companies like Unacademy, Amazon, Practo, and D. E. Shaw.

I hold a Master's in CS from IIIT-Hyderabad, specializing in Databases, Information Retrieval and Web Mining. I teach a cohort-based course on System Design and have taught 1000+ engineers. On the side, I am

    building a tiny in-mem database DiceDB
    running a YouTube channel with 20k subscribers
    publishing a newsletter with 21k readers
    putting out one-liners through The Smarter Chimp
    reading and implementing a bunch of research papers
    reading a few engineering books every now and then

If you're capable of performing at the L+1 level, start doing so - do not wait to be promoted or compensated for it first. You are promoted, because you have already started taking on the responsibilities of the next level.

Arpit Bhayani

creator DiceDB • Staff Engineer at Google Ads • Founder Profile.fyi • ex-Amazon Fast Data • ex-DoE Unacademy • Sparking engineering curiosity by writing about System Design and Career Growth • Angel Investor

Taplio

Arpit Bhayani's Linkedin Analytics

's Best Posts (last 30 days)

Want to drive more opportunities from LinkedIn?

Taplio

.css-1jgreyo{background-image:linear-gradient(284deg, rgba(39,43,72,1) 0%, rgba(33,47,183,1) 100%);color:transparent;-webkit-background-clip:text;background-clip:text;}Arpit Bhayani's Linkedin Analytics

's Best Posts .css-qqfgvy{font-size:var(--chakra-fontSizes-md);}(last 30 days)

Want to drive more opportunities from LinkedIn?

Arpit Bhayani's Linkedin Analytics

's Best Posts (last 30 days)