TalentNeuron

Leading the team of Machine Learning developers digging through the gigabytes of talent marketplace data.

Tools and techniques used: machine learning (deep/shallow, LLMs, classification, generative AI, clustering, topic modeling), regular expressions, Python, MySQL, scikit-learn, PyTorch.

Machine Learning Lead

True Positive Inc.

“If you intend to use machine learning to solve business problems at scale, I'm delighted you got your hands on this book.”

—Cassie Kozyrkov, Chief Decision Scientist at Google

“Foundational work about the reality of building machine learning models in production. Comes at the right time when companies start to see through the AI hype, realizing it takes a conscious engineering effort and best practices to make machine learning work. Another great book from Andriy!”

—Karolis Urbonas, Head of Machine Learning and Science at Amazon

Released on Amazon, B&N, and Leanpub. Drafts are available on http://www.mlebook.com.

Author of the Machine Learning Engineering book

Peter Norvig, Research Director at Google, co-author of AIMA, the most popular AI textbook in the world: "Burkov has undertaken a very useful but impossibly hard task in reducing all of machine learning to 100 pages. He succeeds well in choosing the topics — both theory and practice — that will be useful to practitioners, and for the reader who understands that this is the first 100 (or actually 150) pages you will read, not the last, provides a solid introduction to the field."

Aurélien Géron, Senior AI Engineer, author of the bestseller Hands-On Machine Learning with Scikit-Learn and TensorFlow: "The breadth of topics the book covers is amazing for just 100 pages (plus few bonus pages!). Burkov doesn't hesitate to go into the math equations: that's one thing that short books usually drop. I really liked how the author explains the core concepts in just a few words. The book can be very useful for newcomers in the field, as well as for old-timers who can gain from such a broad view of the field."

CEO and Author of The Hundred-Page Machine Learning Book

Gartner

Leading a team working on "hacking" text and other unstructured or semi-structured data to extract useful information. Problems successfully solved by me and my team include:

— conversational user interfaces (chatbots);
— parsing of free-form résumé to detect and extract candidate name, employers, skills, certifications, employment history;
— parsing of job descriptions to detect and extract job title, salary, employer name, assign a standard occupation code;
— automated noisy and multi-language data normalization, cleanup, deduplication;
— novel data detection;
— language detection / text segmentation by language;
— sequence labeling (queries, addresses, salaries);
— machine translation;
— Web-data analysis.

Tools and techniques used: machine learning (deep/shallow, classification, clustering, topic modeling, structured output prediction), regular expressions, Java, Python, MySQL, scikit-learn, Keras.

Director of Data Science — Machine Learning Team Leader

Wanted Technologies

Turning terabytes of job announcements from online job boards into structured knowledge: extraction and classification of skills, standardization of job titles, classification of job announcements by an occupational classification, extraction/cleanup of structured data (company names, proper names, salaries, skills, etc), extraction and standardization of postal addresses, fuzzy duplicate identification (announcements, skills).

Tools: linux shell scripting, MySQL, Sphinx, Python (scikit-learn, NLTK), Java.

Data Analyst

Fujitsu Canada

Computer scientist with the focus on applying machine learning and natural language processing in the context of big data; technology intelligence lead.

Technologies: R, Python, Hadoop, HBase.

Applied Scientist — Fujitsu Innovation Center

Working on research contracts for the Defence Research and Development Canada (DRDC).

Research projects mostly turned around the problems of fusion of sensor data and intelligent software assistants.

Technologies: Scilab.

Scientific Consultant

Université Laval

Research in multiagent systems resulted in multiple publications in top international conferences and journals (AAMAS, AAAI, Journal of Algorithms).

Student Researcher

Read my bestselling 📖 The Hundred-Page Machine Learning Book: http://themlbook.com.

Read my new book 📖 Machine Learning Engineering: http://mlebook.com.

Subscribe to my weekly ✉️ Artificial Intelligence newsletter: https://www.linkedin.com/newsletters/artificial-intelligence-6598352935271358464/ and to my weekly ✉️ Data Science newsletter: https://www.linkedin.com/newsletters/7102511020270608384/

About me:

Ph.D. in artificial intelligence, passionate about data, speaking English, French, and Russian. Solid scientific programming and team leadership skills. More than 20 years of experience working on various computing projects, including several own startups.

More than 15 years of practical hands-on experience in automated data analysis, machine learning, natural language processing, Linux hacking, database and search engine setup and querying.

Expert in Python and Java with several years of day-to-day design and development in the big data context.

Specialties: machine learning, natural language processing, conversational interfaces (chatbots), information retrieval.

This is a port in Guangzhou, China. The ships are unloaded remotely by operators via 5G. Then AI vehicles automatically drive the containers to trucks and load them, without human assistance.

Andriy Burkov

ML at TalentNeuron, author of 📖 The Hundred-Page Machine Learning Book and 📖 the Machine Learning Engineering book

"AI-generated content detectors" are a scam. The one below claimed that the US Declaration of Independence is 97% AI-generated. They're all like that because detecting AI-generated content is, even in theory, impossible—unless the content is watermarked in some way. If a teacher accuses your child of using ChatGPT, send them this link: https://lnkd.in/e4sQBdzp

Scientists have mapped the entire neural network of the fruit fly: 50M synapses between 139k neurons.

This is so cool! This is what technology is for.

I see this picture circulating a lot so I cannot just let it fool the members of this highly respected community.

Programmers hate to talk to clients. This is why the industry invented project managers and business analysts. They are safe, not programmers.

Programmers are actually the most unsafe at this moment, because LLM-generated code can be automatically verified for validity by running it and applying to unit tests. Unit tests can also be generated in thousands by an LLM. 

So, instead of denying that the profession is being challenged and sharing funny memes, why wouldn't someone start thinking on how the programmers can transition into doing something LLMs cannot do out of the box? Put your ideas in the comments.

Now their grandkids are protesting against the use of ChatGPT. Change is hard.

While working on my #theLLMbook, I train models to show the result as you go from simpler to more advanced models. So that the reader is capable of reproducing my experiments without spending a fortune on GPUs, I took a relatively small text corpus, the Brown Corpus that has about 1M tokens.

No matter how hard I tried, both RNN and transformer, when trained on this corpus, resulted in a model that generated garbage. I started thinking that there's something fundamentally wrong in how I programmed these two. But two at the same time? That was weird.

So I started reading papers from 2010-2015 where Ilya Sutskever, Andrej Karpathy, and Tomas Mikolov still trained RNNs as language models. Their perplexity was ~120, my was ~280.

And I found the problem. They used a dataset 70 times larger. But they couldn't use it entirely, it was too large for the time (a GPU had 4GB or memory). So they used like 20% of the data. Once I switched the dataset and used it entirely, my perplexity went to ~50 with RNN (one layer) and ~35 with a transformer (two decoder blocks). So, I beat their results by ~2.5 times just by using the full dataset on a modern GPU. (Just for your info, GPT-2 has the perplexity of about 20.)

What's the takeaway? Modern advancements in language models are solely due to huge data and huge GPUs. Did I know that? Yes. But one thing is to know and another thing is to learn it hands on the hard way.

Think about doing a PhD? Don't.

The paper: https://lnkd.in/eJra5Hgb

Finally, I joined the club of those who have trained a transformer decoder language model from scratch in Google Colab! I achieved the perplexity of 67. Which is a 2 times better result than anyone could hope to get 5 years ago, before GPT-2 was released. Look at the continuation for the prompt "The President":

==
The President is a major contributor to the U.S. , and the government has been `` very disappointed '' .
==

The first sign that a language model is improving is its ability to handle quotes and parentheses correctly (i.e., close them after opening and respect phrasal categories).

While this is far from modern LLMs' perplexity of ~5, but as Microsoft Research said about ChatGPT when it was just released, I can see "the sparks of AGI" even at 67.

You should try programming and training your own decoder and watch the model evolving from spitting out random tokens to producing something meaningful. It's a rewarding experience.

Anyone can fine-tune what already works. Very few have trained something from scratch. Be one of the few.

An intern uses ChatGPT to write 80% of their code and impresses two Fortune 500 companies. “None of these ideas were original and generated by my own intellect”.

Parents of LinkedIn: if your child's professor accuses them of using ChatGPT in their work, they are bluffing. Commercial tools claiming to detect AI-generated content are nothing but snake oil. No scientifically proven method exists for detecting LLM-generated content, and given how LLMs are trained, it's even theoretically impossible. So don't let them bully your children—call their bluff.

While writing my book on language models, I realized that for someone who's not a developer, the entry barrier to modern ML is quite high. Not only do you need to learn coding, OOP, and Python, along with at least some regular expressions, but then you have to understand tensors and operations on them in PyTorch. After that, there's the math of matrices, functions, derivatives, and a basic understanding of probabilities and random processes.

And then, if you manage to get these fundamentals clear, you need to learn ML basics like overfitting, generalization, hyperparameters, regularization, early stopping, types of errors, metrics, and countless other tricks. It's crazy!

The EU is using VPN to download Llama 3.2 from HuggingFace, 2024.

I finally made it to the Transformer part in my new book on language modeling. It was tough to get there, but now it's so beautiful that I have to share it! 😍 #theLLMbook

You should only ask an LLM to explain how something works when you know the subject well enough to recognize errors in the explanation. Studying a new subject using an LLM is self-sabotage.

Taplio

Andriy Burkov's Linkedin Analytics

Who is engaging with Andriy

's Best Posts (last 30 days)

Want to drive more opportunities from LinkedIn?

Taplio

.css-1jgreyo{background-image:linear-gradient(284deg, rgba(39,43,72,1) 0%, rgba(33,47,183,1) 100%);color:transparent;-webkit-background-clip:text;background-clip:text;}Andriy Burkov's Linkedin Analytics

Who is engaging with Andriy

's Best Posts .css-qqfgvy{font-size:var(--chakra-fontSizes-md);}(last 30 days)

Want to drive more opportunities from LinkedIn?

Andriy Burkov's Linkedin Analytics

's Best Posts (last 30 days)