Arrow Global Group

Senior Data Engineer

Pytest with Eric

Founder

ADSE Tech Limited

• Reduced ML Model deployment time by 50% using Terraform, GitHub Actions, and Google CloudFunctions over existing manual untraceable deployments.
• Achieved 50% improvement in SQL query execution time by redesigning dimension/fact modelling based on user access patterns for BigQuery Warehouse and fewer joins in DBT.
• Introduced schema versioning and integration tests using PubSub, Pytest, Hypothesis, DBT and SQL to improve study plan module generation pipeline uptime by 75%.
• Mentored data scientists and put complex ML LLM models into production with tracking using GCP Cloud Run and Vertex AI (e.Neo4j-based word and sentence generator).

Senior Data / MLOps Consultant (Client Based)

Coral

• Created end-end automated data integration pipelines for over 20 different sources (FB, IG, Linkedin,
TikTok) using AirByte on Kubernetes.
• Decreased ETL Pipeline execution time by 20% using scalable and efficient DBT data modeling.
• Increased ETL Pipeline uptime by 50% with Unit and Integration tests using Pytest, Hypothesis, DBT
and GitHub Actions.
• Designed dimension/fact modeling for over 100 tables in Snowflake, Redshift and BigQuery.
• Wrote reliable, scalable and easy-to-maintain Spark, Glue, Python and Kubernetes jobs to process
hundreds of thousands of rows via serverless and non-serverless production-grade setups.
• Designed & developed a job orchestration system using Step Functions, DynamoDB, Eventbridge, Mage and Airflow achieving 50% better execution tracking/restart than the existing orchestration solution.
• Architected a secure auditable data lake using S3, Gue, and Lakeformation with best practices, access control, data lineage and metadata.
• Secured Snowflake environments by using Terraform to manage Snowflake users and role privileges as a major improvement over current manual accesses.

Senior Data & Backend Engineer (Contract)

Nutrivy

• Started Health Supplement E-Commerce Company. Built to 5-figure revenue, sold in December 2021.
• Set up custom product and sales metric tracking dashboards with Python, Metabase and RDS.

Adarga

• Improved performance of real-time NLP Microservices by 20-25% (efficient data structures, ephemeral
storage/caching, parallel processing and multithreading).
• Improved quality of data projected to Elasticsearch by 20% by developing a robust Text Cleaning solution.
• Created a Python-Flask-Docker-Kubernetes CI/CD Pipeline for data scientists to deploy machine learn- ing models into production within 3-5 minutes.
• Influenced new data model & schema for improved performance from Elasticsearch and Kafka.
• Built end-end data pipeline to detect entities from documents in real-time- using Kafka Streams & Kubernetes.
• Improved performance of the overall ETL pipeline by 10% using document chunking in S3 and Kafka.

Senior Data Engineer (Permanent)

NewDay

• Created a robust ETL pipeline to process PII and PCI data - using Python, Dask, Spark, and Docker on
Kubernetes.
• Improved ETL uptime by 10% by setting up ETL orchestration solution using Python & Airflow.

Senior Data Engineer/Python Developer (Contract)

The Plum Guide

• Developed ETL solution to ingest over 100k homes and listings per day – using Spark, on AzureHD-Insight, SQL Server and Snowflake.
• Improved data warehouse capabilities and reduced operational costs by 5% by introducing Snowflake Warehouse

Senior Data Engineer (Contract)

Collinson

• Implemented a production-grade Insurance Price Optimisation API (over 5k requests/sec) with Python, Flask, EMR, RDS and API Gateway.
• Implemented a Real-Time data ingestion pipeline for tracking clicks (500k/day) on Travel Insurance websites using Kinesis, Lambda and Pyspark.
• Improved insurance policy sales by 3% by building an ML-based insurance pricing optimiser and product recommender with EMR, Python, Flask, Spark, and API Gateway.

Big Data and Cloud Infrastructure Engineer (Permanent)

Intelematics

• Built data science models to predict vehicle fault codes and battery failures in advance leading to a 5% reduction in vehicle maintenance costs, using Spark, Kafka, AWS S3, SQS, DynamoDB, and NodeJS.
• Productionized ingestion pipelines and Tableau dashboards for company KPI performance for C-Suite executives – using Tableau, Python, and Redshift.

Data Scientist & Engineer (Permanent)

Senior Data & MLOps Engineer with 7+ years of expertise building 50,000+ User production ETL pipelines and ML microservices across AI, Health Care, Growth Marketing, Financial, EdTech, Insurance and IoT industries (from early-stage start-ups to Series C+).

Taplio

Eric Sales De Andrade's Linkedin Analytics

Eric Sales De Andrade

Want to drive more opportunities from LinkedIn?

Taplio

.css-1jgreyo{background-image:linear-gradient(284deg, rgba(39,43,72,1) 0%, rgba(33,47,183,1) 100%);color:transparent;-webkit-background-clip:text;background-clip:text;}Eric Sales De Andrade's Linkedin Analytics

Eric Sales De Andrade

Want to drive more opportunities from LinkedIn?

Eric Sales De Andrade's Linkedin Analytics