Serix

Co-Founder

Airflow | Talend| MSSQL| Postgres | Linux|SQL|Python|AWS| Spark |Kafka |Scala |RedShift.
✅ Create and maintain optimal data pipeline architecture.
✅ Assemble large, complex data sets that meet functional / non-functional business requirements.
✅ Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
✅ Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
✅ Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.

Data Engineer

Bidco Africa

✅ Design, implement, and maintain data pipelines that constitute the company data platforms.
✅ Develop and maintain ETL workflows that extract data from various sources, transform it, and load it into our data platform based on business demands.
✅ Mediating and coordinating the resolution of data science project deliverables using agile methodology.
✅ Design, architect, re-architect, and develop BI solutions, and KPIs for different users
✅ Ensure data integrity and accuracy in automated data pipelines.
✅ Engage actively in collaborative meetings with multidisciplinary teams to deliberate on objectives and apply agile methodologies for project implementation.
✅ Ensuring strict compliance with the Data Protection Act when handling customer data.
✅ Conduct data analysis and provide insights to support decision-making processes.
✅ Automate tasks (Business Processes).

Lead Data Scientist

Data Scientist

ProjectPro

Creator

StepWise

✓Worked On ComputerVision and Machine Translation Projects.

Omdena

✅ Collaborated with 30 AI Engineers Globally to help PolicyMakers Improve Food Security, and Crop Yield in Malaysia. Crop yield prediction using Ml.
✅ Developed dashboards incorporating crop detection using satellite imagery, Crop yield Prediction, Crop disease, and Climate risk prediction solutions.

Machine Learning Engineer

afringa

Data Science Intern

Forage

✓Acquired more intuition as I walked through the following modules In Sprocket Company limited data implementation:
       ✓.Data Insights
       ✓.Data Insights and Presentation
       ✓.Data Quality Assessment.

KMPG Data Analytics Consulting  Internship

Data Scientist | Data Engineer| Cloud Architect|

As a skilled Data Scientist, and Engineer, I am dedicated to building Data Products that tackle real-world issues in various industries such as Finance, Agriculture, Health, Education, Manufacturing, and Telecommunication. My expertise lies in designing robust data pipeline architectures and automating ETL processes to help organizations make sense of disparate data and resolve complex challenges.

I am proficient in various programming languages and tools such as Python, Apache Airflow, Spark, Kafka, SQL, Java, Excel, ETL (Talend), PowerBI, Tensorflow, Natural Language Processing, Git, and Version Control. Additionally, I have experience working with Flask, Streamlit, Heroku, AWS Beanstalk, EC2, EMR, RDS, Redshift, relational SQL, and NoSQL databases, including PostgreSQL, SQL Server, OLTP, and OLAP data processing systems.

My expertise extends to DevOps using Docker, Kubernetes, and Data Warehousing, enabling me to deliver top-notch solutions that meet organizational objectives.

Top 6 Tools to Turn Code into Beautiful Diagrams 
. 
. 
- Diagrams 
- Go Diagrams 
- Mermaid 
- PlantUML 
- ASCII diagrams 
- Markmap 
 
Over to you: Did we miss anything? What's your favorite? 
 
– 
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/3KCnWXq 
 
#systemdesign #coding #interviewtips 
.

Wanjohi  Christopher

24 TED Talks to Transform Your Life in 2024:

The Best TED collection for improving key skills covering:

- Well-being
- Leadership
- Personal development

You’ll even get a dose of humor.

1) Jane McGonigal: How to gain 10 extra years of life 
https://lnkd.in/dUtVpE5M

2) Kelly McGonigal: How to embrace stress
https://lnkd.in/dckzvvcF 

3) Larry Smith: Why you won’t have a great career
https://lnkd.in/duKQiXFa 

4) Dan Harris: Don’t be a jerk to yourself
https://lnkd.in/dAUeEAun 

5) Tim Urban: The mind of a master procrastinator
https://lnkd.in/dATYAGBH 

6) Susan Cain: Celebrating introverts
https://lnkd.in/dXeJNb2i 

7) Elizabeth Gilbert: Find your creative genius
https://lnkd.in/ddv5iqKD 

8) Amy Cuddy: Master your body language
https://lnkd.in/dN2hqgbu 

9) Mariano Sigman & Dan Ariely: Make good decisions as a group
https://lnkd.in/d3yAyFVW 

10) Robert Waldinger: What can we learn from the longest study on happiness?
https://lnkd.in/dc8nMBy7

11) Matt Cutts: Start something new for 30 days
https://lnkd.in/d99dUGbC 

12) Sam Berns: How to live a happy life
https://lnkd.in/dUSsWmW9 

13) Dan Ariely: Do we make rational decisions?
https://lnkd.in/d7k8ZXG8 

14) David Steindl-Rast: The key to happiness
https://lnkd.in/d_g673KZ 

15) Shawn Achor: Work and happiness
https://lnkd.in/dDPmsKdG 

16) Chimamanda Ngozi Adichie: A single story is dangerous
https://lnkd.in/dd9aexha 

17) Ray Dalio: Let the best ideas win
https://lnkd.in/dvscqN-c 

18) David Blaine: Holding my breath for 17 minutes
https://lnkd.in/dRTKUUxW 

19) Celeste Headlee: How to converse better
https://lnkd.in/dF6km_TN

20) Simon Sinek: Be an inspirational leader
https://lnkd.in/dAnG_Jnp 

21) Brené Brown: Vulnerability is good
https://lnkd.in/dZPJN9PT 

22) James Veitch: What happens when you reply to spam email?
https://lnkd.in/dYwugezX 

23) Julian Treasure: Make people want to listen
https://lnkd.in/dyM8rWcb 

24) Jill Bolte Taylor: What my stroke taught me
https://lnkd.in/dNC-SPq8 

♻️ Repost this to help others in your network.
And follow Eric Partaker for more content like this.

📌 Want the hi-res PDF with clickable links?
Try my newsletter: https://lnkd.in/dF-VwKqi
You'll also get 40+ more high-value infographics for free.

Maybe what's holding you back is your perspective?? Thoughts? Comment below 👇🏽

🌍 A Historical Perspective of Our Global #CarbonFootprint (not in in cumulative terms)

Data from Our World in Data is behind this new chart that provides a comprehensive look at the world's top 15 carbon dioxide (#CO₂) emitters from 1850 to 2022.

✅  The Industrial Revolution marked the beginning of significant carbon emissions, with the 🇬🇧  leading as the top emitter.
✅ The 🇺🇸  emerged as a major contributor following its industrial expansion post-Civil War.
✅ Since 2006, 🇨🇳 became the largest emitter, reflecting broader economic and industrial growth trends worldwide. 

Notably, sectors like aviation and shipping also feature prominently in recent rankings.

This historical and contemporary analysis shows that while the biggest economies have historically been the major emitters, the challenge of reducing carbon emissions is a shared global responsibility. 

As highlighted in the World Meteorological Organization’s latest #StateoftheClimate report, our global climate continues to change at an unprecedented rate, underscoring the urgent need for collective #climateaction.

https://lnkd.in/eRjX2XEq

I love Data Analytics! 

As a Data Engineer for the past 9 years I find myself loving the work of Data Analytics more and more. I think there is actually a lot of crossover and overlap to Data Engineer as well.

If you can learn how to create clear dashboards from SQL queries it adds so much value. Most companies aren't dealing with petabytes of data so keeping things simple and working with things like Tableau, Excel, or Power BI is a fantastic skill to learn.

Then working on metrics to guide business decisions and measure impact with experiments is the next step.

I think the rise of Data Analysts + Data Engineers mixing to be Analytics Engineers will be a big thing in the future.

Inspired by Alex Freberg

I'm excited to announce the first release of the Surface Area of Rivers and Lakes (SARL) database, an extensive resource that tracks the global permanent and seasonal water surface extents of rivers and lakes from 1984 to 2022. This database opens new avenues for understanding the spatial and temporal dynamics of surface water by waterbody type. Such insights are essential for effective water resource management, flood risk assessment,
ecosystem accounting, and refining biogeochemical cycle estimates. A notable finding from our research is the significant increase in seasonal water extents across numerous rivers and lakes. Furthermore, this research underscores the potential impact and the need for more Global Landform Maps (https://lnkd.in/dFzvRdw6) to better understand the changing characteristics of landscapes over time, benefiting a wide range of disciplines and applications.

Explore the interactive SARL map here -> https://lnkd.in/d4k-BxP3

Article -> https://lnkd.in/dpaTrCS3

GEE Source Code -> https://lnkd.in/dVTWA7eh

#water #waterresourcemanagement #flooding Roger Sayre Elco Luijendijk Realfag UiB 7Analytics

IF YOU FEAR FAILURE, WATCH THIS!

Amazing speech from Deshauna Barber.

A data pipeline that produces data nobody uses is a bill, not an asset! 

Being mindful of the business value and longevity of your work is what separates the good data engineers from the mediocre ones 

#dataengineering

Breaking into data engineering can feel complicated and overwhelming!

You need to learn the languages of the trade SQL and Python.

You need to learn the tools of the trade Spark,BigQuery, Airflow, Databricks, etc. 

Then you need to show that you actually know this stuff!

In this blog post, I go into detail about how to get into data engineering in 2024. You can find the link in the comments!

#dataengineering

Quality data is more important for machine learning than picking the right algorithm!

Here’s how to test if you have quality: 

- the basic tests:
Are there NULLs where there shouldn’t be? 
How about malformed text fields? 
All basic checks should be able to be done with a simple SELECT and WHERE clause 


- the intermediate tests 
Are there any duplicates? 
Does the data obey business logic when JOIN’d to other high quality data sets?
Are there any missing data for critical dimensional cuts? 
All intermediate tests should be able to be solved with SELECT, GROUP BY and JOIN


- the advanced tests
How do row counts look month over month? Is there enough data? 
Are there any anomalous row counts when cutting by important dimensions? 
Advanced tests usually need light machine learning on top of a SQL query otherwise they get noisy!

What other tests do you do on your data?

#dataengineering

A 3-hour #EarthEngine and #geemap workshop at the TNGIC - Tennessee Geographic Information Council Annual Conference

Video: youtu.be/UyhmH_Al9fk
Notebook: https://lnkd.in/ePH6CkF4

youtube.com

Earth Engine and Geemap workshop at TNGIC Annual Conference 2024

AI is a data product.

Gone are the days of creating random chatbots just to say you’re building AI. In 2024, teams are likely to become more sophisticated about how they develop AI products leveraging better training practices to create value instead of pumping out technology to create new problems. 

In AWS’ 2023 CDO survey, respondents were asked what their organization’s biggest challenge was in realizing the potential of Gen AI.

The most common answer?
Data quality.

And like any data product, it doesn’t function without reliable data. At the scale of LLMs, manual monitoring can’t provide the comprehensive quality coverage required to make any AI reliable.

To truly be successful, data teams need a breathing data observability solution tailored to AI stacks that can empower them to detect, resolve, and prevent data downtime.

Here is a prediction for 2024:

Data observability solutions that prioritize 1) resolution, 2) pipeline efficiency, and 3) the streaming/vector infrastructures that support AI will be essential in the modern AI reliability battle.

The above are summaries of Tomasz Tunguz's talk on data observability. 

If you want more insiders take like this, I'm hosting two virtual panels on Data Quality Day, a half-day session with industry leaders on how to land successful AI initiatives in your team. Register now: https://lnkd.in/eDf_jhvT

Share with a friend who might find it useful!

How do you tackle data challenges for AI?

Everyone doing LLMs in production should spend some time on the documentation of TGI - newly apache 2.0 - used by thousands of organizations (including Hugging Face). Here's a quick tour from https://lnkd.in/eWV4BmjM!

Remote Data Engineer Role:

In my earlier post I have shared few companies that hire Data Engineers for remote role: https://lnkd.in/g2nTviPW

Here I am listing out few more companies they hire Remote data engineers.

🔺 Companies and their hiring roles.

👉 Skillsoft - Principal Data Engineer
https://lnkd.in/gaDZZS7n

👉 Oportun - Senior Data Engineer
https://lnkd.in/ghHyWMk4

👉 IOMETE - Data Engineer
https://iomete.com/

👉 Offshore - Celonis - Data Engineer
http://www.rigusinc.com/

👉 Qubrid - Data Engineer 
https://www.qubrid.com/

👉 Appreciate - Data Engineer
https://lnkd.in/ge84v256

👉 Durendal Resources - Databricks Data Engineer
https://lnkd.in/g4sppJGV

👉 Koantek - Senior Data Engineer
https://www.koantek.com/

👉 Vibrent health India - Senior Data Engineer
https://lnkd.in/gJaYYje5

👉 TASC Outsourcing - Senior Data Engineer (iPaaS)
https://lnkd.in/gHzb7rQY

👉 Edgio - Data Engineer - Data Warehouse and Analytics
https://lnkd.in/gnW3vckX

👉 Spiceworks - Data Engineer II (SQL, Snowflake, AWS, ETL)
https://lnkd.in/gDeC3NcG

👉 Anko GCC - Lead, Data Engineer
https://lnkd.in/gGQfqB-V

👉 CrowdStrike - Sr. Data Engineer (Remote, IND)
https://lnkd.in/gxw2YGuH

👉 WIN Home Inspection - Senior Data Engineer
https://lnkd.in/gygmJKfQ

👉 Procore Technologies - Senior Data Engineer
https://lnkd.in/gyZxuP7u

👉 Argano - Azure Data Engineer
https://argano.com/

👉 Tide - Senior Data Engineer(Snowflake/DBT)
https://www.tide.co/

👉 The Smart Cube - TSC - Manager - Data Engineer 
https://lnkd.in/gkAsHZry

👉 TELUS International Digital Solutions - GCP Data Engineer
https://lnkd.in/ggRKHa78

👉𝐅𝐨𝐥𝐥𝐨𝐰 Abhisek Sahu for more data engineering related information and materials. Save and reshare ✅

👉 𝐉𝐨𝐢𝐧 𝐓𝐞𝐥𝐞𝐠𝐫𝐚𝐦 𝐂𝐨𝐦𝐦𝐮𝐧𝐢𝐭𝐲 - Data & Cloud Engineering Professionals - https://lnkd.in/gy4R55Tj

#remotework #dataengineering #databricks #bigdata #hiring #wfhjobs #linkedin #jobs #sql

Two years ago I left Facebook to go all in on myself. I left the golden handcuffs despite having a great team and a great job but I decided to leave.

It was scary not knowing where the next paycheck would come from. I had a few clients, but it was still unnerving.

Since then I have nearly tripled the size of my business, learned a whole new set of skills, and got to work on a bunch of fun projects.

If you're looking to start your own data analytics or software consulting company or maybe you just want to take on the occasional side hustle, then I put together a guide to help you do it!

It covers:

- Starting your consulting company
- Finding your niche
- Finding clients through marketing, sales, and other methods
- Books you should read
- and more!

If you enjoyed this post, then feel free to leave a like and re post(side note, this repo just passed 1k stars)!

Thank you to Zach Wilson for the inspiration.

github.com • 1 min read

GitHub - sdg-1/consulting-handbook: A guide for technical professionals looking to start consulting

Companies I've worked with that have successfully used data, don't generally try to roll-out all their data infrastructure at once. They aren't trying to implement AI and ML day one or trying to create the most awe-inspiring technical diagram.

Instead, they often go through a few phases. Each phase helping them grow both on the technical as well as business side of using data.

Here are the general phases I have seen companies go through to build a reliable data stack.

1. Spreadsheets - In this phase likely an analyst or operations lead is just asking a software engineer every week for a data dump for them to look at data. Both the size of the data and the frequency of these requests make it unnecessary to automate

2. Replica Database - At this stage, the data you're looking at might be a little too large for spreadsheets to handle and you might need that data updated more often. Thus a replica database might not be a bad next phase especially if you don't have the budget for a data engineering team.

3. Basic ETL + Data Warehouse - Eventually, the software team is too distracted by the product they are trying to build to continue to create data sets for the analysts and likely your company is growing rapidly. Meaning there are a lot more data sources that need to be centralized. So you'll likely hire a data engineer to set up your baseline data warehouse.

4. Adding Complexity Where Required - Naturally end-users will want to have more complex pipelines created. Maybe they'll start talking about AI and ML or some more complex business logic. Now that the company has experience deploying basic ETLs it can start to make sense to talk about.

5. Operationalizing Data - Finally, once your company has become confident in how it processes and manages data, it'll likely look to re-incorporate that data back into the product and workflows that directly impact the business.

Are there any other phases you've seen?

Also, if your team is looking for help with your data infrastructure or strategy, then feel free to find some time with me below on the calender link.

🤩 Exciting news! The AWS Community Builders Dashboard is live! ⬇

URL: https://lnkd.in/dYPXC-uA

This dashboard will show you some statistics about the Amazon Web Services (AWS) program and how many CBs there are for each category, country and cohort.

What you can do:
🌟 On the map you can click each number and it will show the country name and a pie chart with the % of community builders for each category in that country
🌟 On the top left of the screen there is an orange arrow that opens a menu:
   - You can change colors of the graphs
   - You can change from Dark mode to White mode

Some recommendations and considerations before use:
🔎  Zoom out (like 80%) the page to get a full overview of all graphs
💻  Recommend to use big screen/monitor
🗺  Map goes full screen in Safari but not in Chrome
⏳  Map loads very slowly in Safari and in Chrome is faster
⏲  First time to execute the dashboard it takes some time to download all files

I will soon post a blog about how I did it and also the source code.

Thanks to Jason Dunn for providing me with the latest data available from the CBs program.

⬇ Let me know what you think in the comments!

Hope you like it! ❤️

✨ ✨ Super excited to share that I will be speaking in the inaugural AWS Community Day Kenya alongside Clive Kamaliki at KCA University on the topic IoT & Sustainability! 🚀🚀

Join us 📆 20th April, 2024 at 📍 KCA University as we delve into the transformative power of IoT in shaping the future of sustainability. From smart homes to industrial automation, IoT is revolutionizing how we live, work and interact with technology.

In the session, we'll explore: 
🔷The fundamentals of IoT 
🔷 IoT and sustainability 
🔷 Opportunities and Challenges in implementing IoT solutions 

Whether you're a seasoned professional or just diving into the world of IoT, this session will offer valuable insights in the IoT landscape.

We are thrilled to connect with industry peers, thought leaders and enthusiasts to exchange ideas and perspectives. Let's unpack the potential of IoT together! 💡

Registration link 🖇️: https://lnkd.in/g7CwwmUJ

AWS User Group Kenya Adeline Makokha  Kevin Tuei Nelly Nyadzua Mark Orina Pauline Namwakira T.S.Khamala Chibire 


#AWSCommunityDayKE
#AWSCommunityDayKE2024
#IoT #InternetOfThings #Technology #Innovation #Sustainability #Networking #DigitalTransformation #awscommunity

NEW DATA: CTrees’ canopy height map for California in 2020 has been published on Amazon Web Services (AWS) Open Registry, a free catalog of more than 3,000 datasets from data providers across sectors. The CTrees data is now available as cloud-optimized geotiff files for any use.  

➡ See the registry listing: https://lnkd.in/gs5eQnaN

The open dataset follows a paper published last month in Remote Sensing on Environment from lead author Fabien H Wagner, Ph.D., that detailed a methodology for producing accurate canopy height maps using a deep learning model on very-high-resolution airborne imagery from the U.S. Department of Agriculture’s National Agriculture Imagery Program (NAIP). Wagner is a research scientist at CTrees and a postdoctoral researcher at the UCLA Institute of the Environment and Sustainability. 

CTrees’ data architecture is implemented in AWS, and the organization is a recipient of the AWS 2023 AWS IMAGINE Grant for Nonprofits. The canopy height map is the first CTrees dataset to be published on the AWS Open Registry. CTrees plans to make more of its scientific datasets available on the registry as they become available.

Taplio

Wanjohi Christopher's Linkedin Analytics

Wanjohi Christopher

Wanjohi Christopher's Best Posts (last 30 days)

Want to drive more opportunities from LinkedIn?

Taplio

.css-1jgreyo{background-image:linear-gradient(284deg, rgba(39,43,72,1) 0%, rgba(33,47,183,1) 100%);color:transparent;-webkit-background-clip:text;background-clip:text;}Wanjohi Christopher's Linkedin Analytics

Wanjohi Christopher

Wanjohi Christopher's Best Posts .css-qqfgvy{font-size:var(--chakra-fontSizes-md);}(last 30 days)

Want to drive more opportunities from LinkedIn?

Wanjohi Christopher's Linkedin Analytics

Wanjohi Christopher's Best Posts (last 30 days)