Aurimas Griciūnas

Senior Solutions Architect

SwirlAI

Driving MLOps, Data Engineering and Machine Learning industry forward by helping professionals Level Up in these disciplines.

Founder & CEO

neptune.ai

Chief Product Officer

Head of Product

Driving MLOps industry forward by helping Neptune customers succeed in adopting MLOps best practices.

Transactie Monitoring Nederland (TMNL)

Participated in building of ML Platform capabilities. Mainly focused on AWS Sagemaker and supporting AWS Services.

Senior MLOps Engineer

HomeToGo

- Led a Team of Cloud Engineers.
- Led a Team of Data Platform Engineers.
- Led Cloud Native Transformation Initiative.
- Led infrastructure configuration movement to IaaC (Terraform).

Data Architect / Data Platform & Cloud Engineering Lead

- Led a Team of Data Engineers.
- Led a Team of Data Platform Engineers.
- Initiated and led a successful company wide Cloud Native Transformation.
- Migrated entire Data Platform Stack to Kubernetes. This included:

        - Airflow.
        - Superset.
        - Spark.
        - MLFlow.
        - Flink.
        - Airbyte.
        - DataHub.

- Introduced first Stream Processing applications that moved some of the Data Warehouse Processing stages upstream the pipeline.

Senior Data Engineer / Data Platform Team Lead

- Led a team of Data Engineers.
- Initiated and led a Data Warehouse migration from Redshift to Snowflake effort. It also included a refactoring of the Data Model.
- Oversaw Data migration from a Druid cluster to Redshift and later Snowflake.
- Introduced and rolled out Machine Learning Experiment Tracking System.
- Deployed and maintained first on-prem Kubernetes Clusters in the company.

Senior Data Engineer / Team Lead

- Led a successful implementation and optimisation of a custom Website Activity Tracking Ingestion Pipeline.
- Deployed and rolled out a company-wide Airflow Cluster.
- Led a full migration from Jenkins to Airflow for Data Engineering Pipelines.

Senior Data Engineer

Tesonet

Was responsible for Identifying use cases for Machine Learning and implementing ML Based Products end to end:

- Data Extraction.
- Data cleaning and validation.
- Model training.
- Model Packaging and Deployment.

Participated in designing and building Big Data Ingestion Pipelines:

- Setting up and configuring Kafka Topics.
- Developing Kafka Streams and Spark applications.

Senior Data Scientist/Engineer

Danske Bank

Projects I worked on:

-Affordability model enhancement using transaction data.
-Bringing transaction data into Balance Sheet stress-testing.
-Unlocking potential of transaction data throughout Risk Management function.
-Stress-testing of REA components and dynamic analysis on different credit portfolio levels.
-Process optimization and automation. Preparing models for production placement.

Senior Data Scientist/Analyst

Projects I worked on:

-Retail customer affordability project in Big Data analytics team.
-IFRS9 Life-time probability of default (LTPD) model development.
-IFRS9 Prepayment model development.
-IFRS9 impact on P&L analysis.
-IFRS9 and pre-IFRS9 ECL stress-testing and reporting.
-REA stress-testing and dynamic analysis on different credit portfolio levels.
-Process optimization by both improving old approaches and introducing new technologies.

Analyst

ADB Gjensidige

Was responsible for implementation of Solvency2 regulatory standards and providing required reports.

Actuary

Was responsible for generating business analysis reports for management.

Was responsible for calculation of Technical Reserves for Non-Life insurance products by applying different triangulation methods.

Junior Actuary

Follow me to learn about Data Engineering and ML Systems | Author of SwirlAI Newsletter | Public Speaker

Building an AI Startup in stealth. We strongly believe that the future of AI is agentic, our goal is to help AI developers ship their products faster and in a secure and controlled way. If you are interested in hearing more, drop me a DM.

With over a decade of experience in Data Science, AI/ML Engineering, and MLOps, I’m also dedicated to helping practitioners level up their skills in MLOps, Data Engineering, Machine Learning, and related fields. I share my knowledge in a clear and accessible way to make learning straightforward and actionable.

If you want to partner with SwirlAI and get your product in front of 30K+ AI/ML/Data Engineers and Leaders via Newsletter or 120K+ AI/ML/Data Engineers and Leaders via LinedIn, contact me directly.

Leading Product Strategy.

𝗥𝗔𝗚 (Retrieval Augmented Generation) vs. 𝗖𝗔𝗚 (Cache Augmented Generation).

There has been a lot of buzz surrounding CAG lately. Let’s see what the differences are betweenRAG and CAG:

𝘙𝘈𝘎

These are the steps for implementing generation for naive RAG:

𝟭. Embed a user query to be used for contextual search via vector DBs or move straight to the step 2 if no contextual search is required.
𝟮. If Contextual search is required, query the context store to retrieve relevant context. If it is not required, use other means to search for relevant data
𝟯. Combine original user query with the system prompt that instructs the final answer construction.
𝟰. Enrich the final prompt with external context retrieved in step 2.
𝟱. Return the final answer to the user.

𝘊𝘈𝘎

𝟭. Pre-compute all of the external context into a KV Cache of the LLM. Cache it in memory. This only needs to be done once, the following steps can be run multiple times without recomputing the initial cache.
𝟮. Pass the system prompt including user query and the system prompt with instructions on how cached context should be used by the LLM.
𝟯. Return the generated answer to the user. After this, clear any generations from the cache and keep only the initially cached context. This makes the LLM ready for next generations.

𝘔𝘺 𝘵𝘩𝘰𝘶𝘨𝘩𝘵𝘴:

➡️ While it has only been described in a white paper for the first time, it is not a novel concept. We have been using different variations of CAG since Anthropic and OpenAI introduced Prompt Caching.
❌ While it might sound strong on paper, LLMs continue to suffer in accuracy while working with extensively long context.
❌ In real use cases, especially enterprise, CAG would cause a lot of security issues due to inability to isolate data.
❌ CAG does not work with constantly changing data as KV Cache would need to be continuously recomputed.
✅ CAG is strong when you need to cache reasonable amount of static data that is not sensitive.
✅ Real magic happens when you combine RAG and CAG into a single system. More on it in future posts, stay tuned in!

Have you played with CAG already? Let me know in the comments 👇

#LLM #AI #MachineLearning

Founder & CEO • AI Engineer • Follow me to Learn about AI Systems • Author of SwirlAI Newsletter • Public Speaker

Here is why you need to understand 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗥𝗔𝗚 as an AI Engineer.

Simple naive RAG systems are rarely used in real world applications. To provide correct actions to solve the user intent, we are always adding some agency to the RAG system - it is usually just a bit of it.

It is important to 𝗻𝗼𝘁 𝗴𝗲𝘁 𝗹𝗼𝘀𝘁 𝗶𝗻 𝘁𝗵𝗲 𝗯𝘂𝘇𝘇 𝗮𝗻𝗱 𝘁𝗲𝗿𝗺𝗶𝗻𝗼𝗹𝗼𝗴𝘆 and understand that there is 𝗻𝗼 𝘀𝗶𝗻𝗴𝗹𝗲 𝗯𝗹𝘂𝗲𝗽𝗿𝗶𝗻𝘁 to add this agency to your RAG system and you should adapt to your use case. My advice is to think in systems and engineering flows.

Let’s explore some of the moving pieces in Agentic RAG:

𝟭. Analysis of the user query: we pass the original user query to a LLM based Agent for analysis. This is where:

➡️ The original query can be rewritten, sometimes multiple times to create either a single or multiple queries to be passed down the pipeline.
➡️ The agent decides if additional data sources are required to answer the query.

𝟮. If additional data is required, the Retrieval step is triggered. In Agentic RAG case, we could have a single or multiple agents responsible for figuring out what data sources should be tapped into, few examples:

➡️ Real time user data. This is a pretty cool concept as we might have some real time information like current location available for the user.
➡️ Internal documents that a user might be interested in.
➡️ Data available on the web.
➡️ …

𝟯. If there is no need for additional data, we try to compose the answer (or multiple answers or a set of actions) straight via an LLM.
𝟰. The answer gets analyzed, summarized and evaluated for correctness and relevance:

➡️ If the Agent decides that the answer is good enough, it gets returned to the user.
➡️ If the Agent decides that the answer needs improvement, we try to rewrite the user query and repeat the generation loop.

✅ Remember the Reflection pattern from my last Newsletter article? This is exactly that. 👆

The real power of Agentic RAG lies in its ability to perform additional routing pre and post generation, handle multiple distinct data sources for retrieval if it is needed and recover from failures while generating correct answers.

What are your thoughts on Agentic RAG? Let me know in the comments! 👇

#RAG #LLM #AI

A simple way to explain 𝗔𝗜 𝗔𝗴𝗲𝗻𝘁 𝗠𝗲𝗺𝗼𝗿𝘆.

In general, the memory for an agent is something that we provide via context in the prompt passed to LLM that helps the agent to better plan and react given past interactions or data not immediately available.

It is useful to group the memory into four types:

𝟭. Episodic - This type of memory contains past interactions and actions performed by the agent. After an action is taken, the application controlling the agent would store the action in some kind of persistent storage so that it can be retrieved later if needed. A good example would be using a vector Database to store semantic meaning of the interactions.
𝟮. Semantic - Any external information that is available to the agent and any knowledge the agent should have about itself. You can think of this as a context similar to one used in RAG applications. It can be internal knowledge only available to the agent or a grounding context to isolate part of the internet scale data for more accurate answers.
𝟯. Procedural - This is systemic information like the structure of the System Prompt, available tools, guardrails etc. It will usually be stored in Git, Prompt and Tool Registries.
𝟰. Occasionally, the agent application would pull information from long-term memory and store it locally if it is needed for the task at hand.
𝟱. All of the information pulled together from the long-term or stored in local memory is called short-term or working memory. Compiling all of it into a prompt will produce the prompt to be passed to the LLM and it will provide further actions to be taken by the system.

We usually label 1. - 3. as Long-Term memory and 5. as Short-Term memory.

A visual explanation of potential implementation details 👇

And that is it! The rest is all about how you architect the flow of your Agentic systems. 

What do you think about memory in AI Agents?

#LLM #AI #MachineLearning

----------

Be sure to ♻️ repost if you found the article useful and follow Aurimas if you want to get a daily dose of useful AI related content in your feed!

𝗟𝗟𝗠 𝗢𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆 is becoming the key foundation without which reliable Agentic systems can not be built. 𝗧𝗿𝗮𝗰𝗶𝗻𝗴 sits at the core of it, why is it important?

Tracing and instrumentation of software have been around for decades now. With AI systems resembling regular software even more, we are now moving the practice here as well (with a few key differences).

Let’s look into the process of tracing from a perspective of a naive RAG system.

𝘍𝘦𝘸 𝘥𝘦𝘧𝘪𝘯𝘪𝘵𝘪𝘰𝘯𝘴:

𝘼) A Controller in the GenAI system application is the central piece of software that orchestrates the end-to-end process. Think of apps using LangChain, LlamaIndex or Haystack.
𝘽) Trace is the end-to-end application flow from the entry point till the answer is produced, it is composed of smaller pieces called spans.
𝘾) Span is a smaller piece of the application flow that represents an atomic action like a function call or a database query. They can be sequential, or run in parallel.

ℹ️ As part of span we capture general metadata like start and end time, inputs and outputs of the span. On top of this metadata we track information specific to the GenAI system elements.

What might a trace look like for a naive RAG system?

𝟭. A query that has been submitted to the chat application.
𝟮. The query is embedded into a vector. 

✅ Additional metadata like input token count is persisted with the span so that we can estimate the cost of the procedure.

𝟯. ANN lookup performed against the Vector DB to retrieve the most relevant context. 

✅ Additional metadata about the query is persisted as part of the span together with the retrieved pieces of context and their relevance.

𝟰. A prompt is constructed from the system prompt and retrieved context.
𝟱. The prompt is passed to the LLM to construct the answer. 

✅ Additional metadata about input and output token count is captured together with the span so that we can estimate the cost of the procedure.

𝘞𝘩𝘺 𝘪𝘴 𝘵𝘳𝘢𝘤𝘪𝘯𝘨 𝘰𝘧 𝘎𝘦𝘯𝘈𝘐 𝘴𝘺𝘴𝘵𝘦𝘮𝘴 𝘪𝘮𝘱𝘰𝘳𝘵𝘢𝘯𝘵?

- These applications are usually complex chains, errors can happen in different steps of your application. E.g. Embedding of query is taking longer than expected or you have reached API limits of LLM provider.
- Cost for calling LLM APIs will be variable depending on the length of inputs and produced outputs. You would usually trace this information and analyze it to help forecast expenses.
- GenAI systems are non-deterministic and will deteriorate over time. They need to be evaluated on span level rather than input/output of the entire system so that you can tune each piece separately.
- …

Are you tracing GenAI systems? Let me know in the comments 👇

#LLM #AI #LLMOps

As an 𝗔𝗜 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿 you should also care about regular non LLM based ML models, productionising them comes with its own challenges. For example, 𝗖𝗜/𝗖𝗗 process is 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁 𝗳𝗼𝗿 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗣𝗿𝗼𝗷𝗲𝗰𝘁𝘀 compared to regular software.

The important difference that the Machine Learning aspect of the projects brings to the CI/CD process is the treatment of the Machine Learning Training pipeline as a first class citizen of the software world.

➡️ CI/CD pipeline is a separate entity from Machine Learning Training pipeline. There are frameworks and tools that provide capabilities specific to Machine Learning pipelining needs (e.g. KubeFlow Pipelines, Sagemaker Pipelines etc.).
➡️ ML Training pipeline is an artifact produced by Machine Learning project and should be treated in the CI/CD pipelines as such.

What does it mean? Let’s take a closer look:

Regular CI/CD pipelines will usually be composed of at-least three main steps. These are:

𝗦𝘁𝗲𝗽 𝟭: Unit Tests - you test your code so that the functions and methods produce desired results for a set of predefined inputs.

𝗦𝘁𝗲𝗽 𝟮: Integration Tests - you test specific pieces of the code for ability to integrate with systems outside the boundaries of your code (e.g. databases) and between the pieces of the code itself.

𝗦𝘁𝗲𝗽 𝟯: Delivery - you deliver the produced artifact to a pre-prod or prod environment depending on which stage of GitFlow you are in.

What does it look like when ML Training pipelines are involved?

𝗦𝘁𝗲𝗽 𝟭: Unit Tests - in mature MLOps setup the steps in ML Training pipeline should be contained in their own environments and Unit Testable separately as these are just pieces of code composed of methods and functions.

𝗦𝘁𝗲𝗽 𝟮: Integration Tests - you test if ML Training pipeline can successfully integrate with outside systems, this includes connecting to a Feature Store and extracting data from it, ability to hand over the ML Model artifact to the Model Registry, ability to log metadata to ML Metadata Store etc. This CI/CD step also includes testing the integration between each of the Machine Learning Training pipeline steps, e.g. does it succeed in passing validation data from training step to evaluation step.

𝗦𝘁𝗲𝗽 𝟯: Delivery - the pipeline is delivered to a pre-prod or prod environment depending on which stage of GitFlow you are in. If it is a production environment, the pipeline is ready to be used for Continuous Training. You can trigger the training or retraining of your ML Model ad-hoc, periodically or if the deployed model starts showing signs of Feature/Concept Drift.

Let me know your thoughts. 👇

#AI #LLM #LLMOps

What is 𝗥𝗲𝗔𝗰𝘁 𝗔𝗴𝗲𝗻𝘁 pattern?

As with most of the Agentic System Design patterns, it is all about flow engineering - how you construct the initial prompt and how you route your system according to the outputs produced by the LLM.

ReAct stands for Reasoning and Action. This means that the Agent is prompted to do both, reason about the actions it should take next and is provided the tools that it can use to act on the plan in the real world. After the actions are taken, they are used to reason about the next steps.

The implementations can vary, but on a high level you will see:

𝟭. The initial prompt is very important and is something that you would spend time tuning, on a high level:

➡️ You ask the LLM to solve the provided user query.
➡️ Provide the Agent with the list of tools it can use.
➡️ Ask it to print out the Reasoning and results of the Actions taken while performing a step.
➡️ Provide a list of actions and produced results so far - you start with an empty list.
➡️ You prompt the agent to ot take no more than N reasoning loops.

𝟮. This is an optional step where you can analyse the reasoning, you either check if the User Query was solved already or you can do that in the initial prompt. Here is also where you can break out of the loop if exceeding max steps.
𝟯. Reasoning output will have the next tool to be used. You execute the tool call and get the results.
𝟰. Combine the reasoning and tool call result, update the Reasoning and Action history with the data and run the updated prompt through the LLM again.
𝟱. Repeat until either the User Query is answered or N steps are reached.

ℹ️ Similar types of architectures are used to implement fully autonomous agents and agentic systems. The goal is to go beyond just CoT reasoning and allow the agent to reason continuously on actions it is taking in the real world.

❗️ While it is exciting to explore the capabilities of such architectures, it hasn’t been shown yet that something like this would produce consistently accurate results and would be viable in production.

✅ It is still exciting to see how the space of autonomous agents will evolve in the following years!

Have you tried implementing the ReAct agent pattern in your applications? What were your main observations? Let’s discuss in the comment section 👇

#LLM #AI #MachineLearning

The rise of 𝗔𝗜 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴.

What is this role really about?

I’ve recently outlined my thoughts in an episode of SwirlAI Newsletter.

In my opinion the role best fits generalists that posses a blend of skills from:

➡️ AI Researcher.
➡️ ML Engineer.
➡️ Software Engineer.
➡️ (Potentially) Data Engineer.

Unfortunately, it is naive to expect professionals to have fluency in all of them. My short term prediction is that most of the specialists in the role will have a blend of two, falling into one of the categories 𝟭. to 𝟯.

Surprisingly, there is a lot of non-engineering related work in your day-to-day life as an AI engineer. The organisations are still on a hype train around LLMs, and you would be pushed to implement the technology wherever, even if there is no good fit. One of your key responsibilities will be to coach the organisation and be at the forefront of deciding where and if GenAI is necessary to solve a business problem.

AI Engineer is positioned to be the hottest role in the upcoming years. The salaries are high and the demand will keep increasing due to the shortage of talent in the field. 2025 will be the year of agents, 2026 - very likely the year of multi agent systems and autonomous agents.

You can find the article here: https://lnkd.in/djzjR3Hd

I put my thoughts on:

➡️ Why the new role is justified. I ground the reasoning in how LLM based apps have evolved in recent years.
➡️ What it would take to become an AI Engineer.
➡️ The future of the role.

❗️ Bold prediction: most of the engineers will be developing full stack in ~2 years. Every one of them will need to possess skills of what we call an AI Engineer today.

Let me know your thoughts about the role! 👇

#AI #LLM #MachineLearning

𝗗𝗮𝘁𝗮 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲𝘀 𝗶𝗻 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗦𝘆𝘀𝘁𝗲𝗺𝘀 can become complex and for a good reason 👇

It is critical to ensure Data Quality and Integrity upstream of ML Training and Inference Pipelines, trying to do that in the downstream systems will cause unavoidable failure when working at scale.

It is a good idea to start thinking about the quality of your data at the point of creation (the producers). This is where you can also start to utilise Data Contracts.

Example architecture for a production grade end-to-end data flow:

𝟭: Schema changes are implemented in version control, once approved - they are pushed to the Applications generating the Data, Databases holding the Data and a central Data Contract Registry.

[𝗜𝗺𝗽𝗼𝗿𝘁𝗮𝗻𝘁]: Ideally you should be enforcing a Data contract at this stage, when producing Data. Data Validation steps down the stream are Detection and Prevention mechanisms that don’t allow low quality data to reach downstream systems. There might be a significant delay before you can do those checks, causing irreversible corruption or loss of data.

Applications push generated Data to Kafka Topics:

𝟮: Events emitted directly by the Application Services.

👉 This also includes IoT Fleets and Website Activity Tracking.

𝟮.𝟭: Raw Data Topics for CDC streams.

𝟯: A Flink Application(s) consumes Data from Raw Data streams and validates it against schemas in the Contract Registry.
𝟰: Data that does not meet the contract is pushed to Dead Letter Topic.
𝟱: Data that meets the contract is pushed to Validated Data Topic.
𝟲: Data from the Validated Data Topic is pushed to object storage for additional Validation.
𝟳: On a schedule Data in the Object Storage is validated against additional SLAs in Data Contracts and is pushed to the Data Warehouse to be Transformed and Modeled for Analytical purposes.
𝟴: Modeled and Curated data is pushed to the Feature Store System for further Feature Engineering.
𝟴.𝟭: Real Time Features are ingested into the Feature Store directly from Validated Data Topic (5).

👉 Ensuring Data Quality here is complicated since checks against SLAs is hard to perform.

𝟵: High Quality Data is used in Machine Learning Training Pipelines.
𝟭𝟬: The same Data is used for Feature Serving in Inference.

Note: ML Systems are plagued by other Data related issues like Data and Concept Drifts. These are silent failures and while they can be monitored, we don’t include it in the Data Contract.

Let me know your thoughts! 👇

#MachineLearning #DataEnginering #AI

Building a 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝗴𝗿𝗮𝗱𝗲 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 (𝗥𝗔𝗚) 𝗯𝗮𝘀𝗲𝗱 𝗔𝗜 𝘀𝘆𝘀𝘁𝗲𝗺 is a difficult task. Don't miss the 𝘏) and read until the end to understand why 👇

Here are some of the moving parts in the RAG based systems that you will need to take care of and continuously tune in order to achieve desired results:

𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹:

𝘍 ) Chunking - how do you chunk the data that you will use for external context.

- Small, Large chunks.
- Sliding or tumbling window for chunking.
- Retrieve parent or linked chunks when searching or just use originally retrieved data.

𝘊 ) Choosing the embedding model to embed and query and external context to/from the latent space. Considering Contextual embeddings.

𝘋 ) Vector Database.

- Which Database to choose.
- Where to host.
- What metadata to store together with embeddings.
- Indexing strategy.

𝘌 ) Vector Search

- Choice of similarity measure.
- Choosing the query path - metadata first vs. ANN first.
- Hybrid search.

𝘎 ) Heuristics - business rules applied to your retrieval procedure.

- Time importance.
- Reranking.
- Duplicate context (diversity ranking).
- Source retrieval.
- Conditional document preprocessing.

𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻:

𝘈 ) LLM - Choosing the right Large Language Model to power your application.

✅ It is becoming less of a headache the further we are into the LLM craze. The performance of available LLMs are converging, both open source and proprietary. The main choice nowadays is around using a proprietary model or self-hosting.

𝘉 ) Prompt Engineering - having context available for usage in your prompts does not free you from the hard work of engineering the prompts. You will still need to align the system to produce outputs that you desire and prevent jailbreak scenarios.

And let’s not forget the less popular part:

𝘏) Observing, Evaluating, Monitoring and Securing your application in production!

What other pieces of the system am I missing? Let me know in the comments 👇

#LLM #AI #MachineLearning

----------

Be sure to ♻️ repost if you found the article useful and follow Aurimas if you want to get a daily dose of useful AI related content in your feed!

Why is 𝗠𝗼𝗱𝗲𝗹 𝗥𝗲𝗴𝗶𝘀𝘁𝗿𝘆 key in your AI/ML/LLM(Ops) pipeline?

Let's look again into the regular Machine Learning Model Training Lifecycle.

𝗟𝗲𝘁’𝘀 𝗿𝗲𝘃𝗶𝗲𝘄 𝘁𝗵𝗲 𝗺𝗼𝗱𝗲𝗹 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝘀𝘁𝗲𝗽𝘀:

1: Version Control: Machine Learning Training Pipeline is defined in code, once merged to the main branch it is built and triggered.
2: Feature Preprocessing: Features are retrieved from the Feature Store, validated and passed to the next stage. Any feature related metadata that is tightly coupled to the Model being trained is saved to the Experiment Tracking System.
3: Model is trained and validated on Preprocessed Data, Model related metadata is saved to the Experiment Tracker.
4: If Model Validation passes all checks - Model Artifact is passed to a Model Registry. The model is stored and ready to be packaged.

𝗧𝗵𝗶𝘀 𝗶𝘀 𝗶𝘁 - regardless of what deployment type will follow, the model is stored in The Model Registry. Model Registry is what glues Training and Deployment Pipelines together and this is where handover of the Model Artifact happens.

5: The same model can be packaged as a container for different deployment types by implementing a respective interface. E.g.

- Flink application for Stream Processing Deployment.
- gRPC API for Real Time.
- Scheduled inference on the Batch Serving Feature Store API for Batch.

❗️ It is very similar in the world of LLMs. But more on that in the future posts.

- 𝗧𝗵𝗶𝘀 𝗶𝘀 𝗴𝗼𝗼𝗱 𝗻𝗲𝘄𝘀 - as long as you are training your models using historical data the training pipeline is the same for any type of deployment.
- It’s a different story when it comes to online training!

Now this is a bit different in the era of Large Language Models, but more on it in the upcoming posts. Stay tuned in!

Do you have any battle stories about implementing Model Registries? Let’s discuss in the comment section 👇

#AI #LLM #MachineLearning

What does an 𝗘𝗳𝗳𝗲𝗰𝘁𝗶𝘃𝗲 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗘𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 𝗘𝗻𝘃𝗶𝗿𝗼𝗻𝗺𝗲𝗻𝘁 look like? 
 
MLOps practices are there to improve Machine Learning Product development velocity, the biggest bottlenecks happen when Experimentation Environments and other infrastructure elements are integrated poorly. 
 
Let’s look into the properties that an effective Experimentation Environment should have. As a MLOps engineer you should strive to provide these to your users and as a Data Scientist, you should know what you should be demanding for. 
 
𝟭: Access to the raw data. While handling raw data is the responsibility of Data Engineering function, Data Scientists need the ability to explore and analyze available raw data and decide which of it needs to be moved upstream the Data Value Chain (2.1). 
 
𝟮: Access to the curated data. Curated data might be available in the Data Warehouse but not exposed via a Feature Store. Such Data should not be exposed for model training in production environments. Data Scientists need the ability to explore curated data and see what needs to be pushed downstream (3.1). 
 
𝟯: Data used for training of Machine Learning models should be sourced from a Feature Store if the ML Training pipeline is ready to be moved to the production stage. 
 
𝟰: Data Scientists should be able to easily spin up different types of compute clusters - might it be Spark, Dask or any other technology - to allow effective Raw and Curated Data exploration. 
 
𝟱: Data Scientists should be able to spin up a production like remote Machine Learning Training pipeline in development environment ad-hoc from the Notebook, this increases speed of iteration significantly. 
 
𝟲: There should be an automated setup in place that would perform the testing and promotion to a higher env when a specific set of Pull Requests are created. E.g. a PR from feature/* to release/* branch could trigger a CI/CD process to test and deploy the ML Pipeline to a pre-prod environment. 
 
𝟳: Notebooks and any additional boilerplate code for CI/CD should be part of your Git integration. Make it crystal clear where a certain type of code should live - a popular way to do this is providing repository templates with clear documentation. 
 
𝟴: Experiment/Model Tracking System should be exposed to both local and remote pipelines. 
 
𝟗: Notebooks have to be running in the same environment that your production code will run in. Incompatible dependencies should not cause problems when porting applications to production. It can be achieved by running Notebooks in containers. 
 
Did I miss something? 👇

#LLM #AI #MachineLearning

ML/LLMOps fundamentals - 𝗖𝗼𝗻𝘁𝗶𝗻𝘂𝗼𝘂𝘀 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 (𝗖𝗧) and what steps are needed to achieve it.

CT is the process of automated ML Model retraining in Production Environments on a specific trigger. Let’s look into some prerequisites for this:

1) Automation of ML Pipelines.

- Pipelines are orchestrated.
- Each pipeline step is developed independently and is able to run on different technology stacks.
- Pipelines are treated as a code artifact.

✅ You deploy Pipelines instead of Model Artifacts allowing Continuous Training In production.
✅ Reuse of components allows for rapid experimentation.

2) Introduction of strict Data and Model Validation steps in the ML Pipeline.

- Data is validated before training the Model. If inconsistencies are found - Pipeline is aborted.
- Model is validated after training. Only after it passes the validation is it handed over for deployment.

✅ Short circuits of the Pipeline allow for safe CT in production.

3) Introduction of ML Metadata Store.

- Any Metadata related to ML artifact creation is tracked here.
- We also track performance of the ML Model.

✅ Experiments become reproducible and comparable between each other.
✅ Model Registry acts as glue between training and deployment pipelines.

4) Different Pipeline triggers in production.

- Ad-hoc.
- Cron.
- Reactive to Metrics produced in Model Monitoring System.
- Arrival of New Data.

✅ This is where the Continuous Training is actually triggered.

5) Introduction of Feature Store (Optional).

- Avoid work duplication when defining features.
- Reduce risk of Training/Serving Skew.

𝗠𝘆 𝘁𝗵𝗼𝘂𝗴𝗵𝘁𝘀 𝗼𝗻 𝗖𝗧:

➡️ Introduction of CT is not straightforward and you should approach it iteratively. The following could be good Quarterly Goals to set:

- Experiment Tracking is extremely important at any level of ML Maturity and the least invasive in the process of ML Model training - I would start with ML Metadata Store introduction.
- Orchestration of ML Pipelines is always a good idea, there are many tools supporting this (Airflow, Kubeflow, VertexAI etc.). If you are not doing it yet - grab this next, also make the validation steps part of this goal.
- The need for a Feature Store will wary on the types of Models you are deploying. I would prioritise it if you have Models that perform Online predictions as it will help with avoiding Training/Serving Skew.
- Don’t rush with Automated retraining. Ad-hoc and on-schedule will bring you a long way.

Let me know your thoughts! 👇

#LLM #MachineLearning #AI

----------

Be sure to ♻️ repost if you found the article useful and follow Aurimas if you want to get a daily dose of useful AI related content in your feed!

Second part of my hands-on 𝗕𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝗔𝗜 𝗔𝗴𝗲𝗻𝘁𝘀 𝗙𝗿𝗼𝗺 𝗦𝗰𝗿𝗮𝘁𝗰𝗵 series is out! It covers the 𝗥𝗲𝗳𝗹𝗲𝗰𝘁𝗶𝗼𝗻 𝗮𝗻𝗱 𝗪𝗼𝗿𝗸𝗶𝗻𝗴 𝗠𝗲𝗺𝗼𝗿𝘆 𝗽𝗮𝘁𝘁𝗲𝗿𝗻𝘀 𝗼𝗳 𝘁𝗵𝗲 𝗔𝗜 𝗔𝗴𝗲𝗻𝘁. We build it without using any LLM Orchestration Framework!

After finishing the project you will learn:

➡️ What the Reflection pattern in AI Agent systems is.
➡️ How it relates to short-term memory.
➡️ Pros and Cons of implementing Reflection.
➡️ How to build an Agent class that is able to implement a Reflection pattern taking the memory into consideration without using any LLM orchestration frameworks.
➡️ Fix some of the hallucinations we were producing in the first part of the series!

✅ Just to remind ourselves why we are doing this: if you are using any orchestration frameworks for agentic applications, you might be abstracted away from how Agentic patterns are actually implemented there. Having clarity of how the systems actually work helps you build up systems thinking, enabling you to craft advanced applications more efficiently.

You can find the article here: https://lnkd.in/dmrSEcV3

I supplement the article with Jupyter notebook and python scripts that you can find in our AI Engineer’s Handbook GitHub repo (links in the article).

Youtube video for the first part is coming next week!

If you’re looking for a short weekend project be sure to take a look!

Let me know if there are any issues following the project. As always, feel free to DM me or leave comments here or under the article 👇

#LLM #AI #MachineLearning

-------

Be sure to ♻️ repost if you found the article useful.
Follow me and hit that bell if you want to get a daily dose of useful AI related content in your feed!

Found this amazing website that explains the 𝗰𝗼𝗺𝗽𝘂𝘁𝗮𝘁𝗶𝗼𝗻 𝘁𝗵𝗮𝘁 𝗵𝗮𝗽𝗽𝗲𝗻𝘀 𝘄𝗵𝗶𝗹𝗲 𝗽𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗻𝗴 𝘁𝗵𝗲 𝗻𝗲𝘅𝘁 𝘁𝗼𝗸𝗲𝗻 𝘃𝗶𝗮 𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 𝗯𝗮𝘀𝗲𝗱 𝗟𝗟𝗠𝘀.

If you have few spare minutes be sure to check it out:

➡️ It runs by downloading GPT-2 to your browser.
➡️ You can ask it to generate the next token for your example text.
➡️ All the computations are displayed in real time as the generation is happening.
➡️ You can zoom in and explore details of each step.

You can find the website here: https://lnkd.in/dfdeC8_U

I find the creativity behind the tool fascinating! Let me know your thoughts 👇

#LLM #MachineLearning #AI

-------

Be sure to ♻️ repost if you found the content useful.
Follow me and hit that bell if you want to get a daily dose of useful AI related content in your feed!

Year 2025 and beyond will be about 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜. But as a practitioner, you need to remember - it is 𝗻𝗼𝘁 𝗼𝗻𝗹𝘆 𝗮𝗯𝗼𝘂𝘁 𝗟𝗟𝗠𝘀.

LLMs are useful in Agentic systems for key functions like:

➡️ Planning.
➡️ Content extraction.
➡️ Content generation.
➡️ Code generation.

This functionality unlocks use cases that were previously impossible for AI. It enables LLMs to act as the brain of the Agent, but there is a lot of complexity hidden behind what we call tools provided to agents. Why?

A tool is anything that can be invoked by a function. One could say that it can be an URL, but that is essentially the same - you can invoke an URL using a function.

There are many types of systems that can be hidden behind tools:

➡️ Simple functions defined in code.
➡️ VectorDBs and other data stores containing context.

ℹ️ It is interesting to note how we hide Retrieval piece of RAG behind a tool when it is implemented for an agentic system.

➡️ Regular Machine Learning model APIs.

ℹ️ This one requires a separate article, but in short - LLMs are not here to replace good old predictive ML models as they will always be more accurate and have lower latencies.

➡️ Other Agents!
➡️ …

Don’t get fooled by simple abstractions!

❗️ Be sure to properly map out and implement observability in your agentic flows so that you can keep them in check.

What kinds of tools do you give your agents? Let’s discuss in the comments 👇

Recently I started a series where I implement Agentic patterns (like tool use) from scratch in order to teach Agentic systems from first principles. you can find the first article here: https://lnkd.in/gDp4aBJC

#AI #MachineLearning #Agents

Understanding and being able to apply 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗺𝗼𝗱𝗲𝗹 𝗖𝗼𝗺𝗽𝗿𝗲𝘀𝘀𝗶𝗼𝗻 will distinguish you as a standout 𝗔𝗜 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿. Here is why 👇

When you deploy ML models to production you need to take into account several operational metrics that are in general not ML related:

👉 𝗜𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗟𝗮𝘁𝗲𝗻𝗰𝘆: How long does it take for your Model to compute inference result and return it.
👉 𝗠𝗼𝗱𝗲𝗹 𝗦𝗶𝘇𝗲: How much memory does your model occupy when it’s loaded for serving inference results.

Both of these are important when considering operational performance and feasibility of your model deployment.

👉 Large models might not fit on a device if you are considering edge deployments.
👉 Latency of retrieving inference results might make business case non feasible. E.g. Recommendation Engines require latencies in milliseconds as ranking has to be applied as the user browses your website or app in real time. Bad news for LLM fans!
👉 …

You can influence both latency and size by applying different Model Compression methods, some of them are:

➡️ 𝗣𝗿𝘂𝗻𝗶𝗻𝗴: this method is mostly used in tree-based and Neural Network algorithms. In tree-based ones we prune leaves or branches from decision trees. In Neural Networks we remove nodes and synapses (weights) while trying to retain ML performance metrics.

✅ In both cases the output is a reduction in the number of Model Parameters and model size.

➡️ 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗗𝗶𝘀𝘁𝗶𝗹𝗹𝗮𝘁𝗶𝗼𝗻: this type of compression is achieved by:

👉 Training an original large model which is called the Teacher model.
👉 Training a smaller model to mimic the Teacher model by transferring knowledge from it, this model is called the Student model. Knowledge in this context can be extracted from the outputs, internal hidden state (feature representations) or a combination of both.
👉 We then use the “Student” model in production.

➡️ 𝗤𝘂𝗮𝗻𝘁𝗶𝘇𝗮𝘁𝗶𝗼𝗻: a most commonly used method that doesn’t have much to do with Machine Learning. This approach uses fewer bits to represent model parameters.

👉 You can apply quantization techniques both during the training and after the models has been already trained.
👉 In regular Neural Networks what is quantized are Model Weights, Biases and Activation Functions.
👉 Most usual quantization is from float to integer (32 bits to 8 bits. There are talks about 1 bit LLMs  nowadays 😅).

➡️ …

❗️ While the above methods do reduce the size of the models, allowing them to be deployed in production scenarios, there is almost always a reduction in accuracy so be careful and evaluate it accordingly.

What methods for reducing model size have you used? What were the main challenges? Let me know in the comment section!

#MachineLearning #LLM #AI

Fundamentals of a 𝗩𝗲𝗰𝘁𝗼𝗿 𝗗𝗮𝘁𝗮𝗯𝗮𝘀𝗲.

With the rise of GenAI, Vector Databases skyrocketed in popularity. The truth is that a Vector Database is also useful for different kinds of AI Systems outside of a Large Language Model context.

When it comes to Machine Learning, we often deal with Vector Embeddings. Vector Databases were created to perform specifically well when working with them:

➡️ Storing.
➡️ Updating.
➡️ Retrieving.

When we talk about retrieval, we refer to retrieving set of vectors that are most similar to a query in a form of a vector that is embedded in the same Latent space. This retrieval procedure is called Approximate Nearest Neighbour (ANN) search.

A query here could be in a form of an object like an image for which we would like to find similar images. Or it could be a question for which we want to retrieve relevant context that could later be transformed into an answer via a LLM.

Let’s look into how one would interact with a Vector Database:

𝗪𝗿𝗶𝘁𝗶𝗻𝗴/𝗨𝗽𝗱𝗮𝘁𝗶𝗻𝗴 𝗗𝗮𝘁𝗮.

1. Choose a ML model to be used to generate Vector Embeddings.
2. Embed any type of information: text, images, audio, tabular. Choice of ML model used for embedding will depend on the type of data.
3. Get a Vector representation of your data by running it through the Embedding Model.
4. Store additional metadata together with the Vector Embedding. This data would later be used to pre-filter or post-filter ANN search results.
5. Vector DB indexes Vector Embedding and metadata separately. There are multiple methods that can be used for creating vector indexes, some of them: Random Projection, Product Quantization, Locality-sensitive Hashing.
6. Vector data is stored together with indexes for Vector Embeddings and metadata connected to the Embedded objects.

𝗥𝗲𝗮𝗱𝗶𝗻𝗴 𝗗𝗮𝘁𝗮.

7. A query to be executed against a Vector Database will usually consist of two parts:

➡️ Data that will be used for ANN search. e.g. an image for which you want to find similar ones.
➡️ Metadata query to exclude Vectors that hold specific qualities known beforehand. E.g. given that you are looking for similar images of apartments - exclude apartments in a specific location.

8. You execute Metadata Query against the metadata index. It could be done before or after the ANN search procedure.
9. You embed the data into the Latent space with the same model that was used for writing the data to the Vector DB.
10. ANN search procedure is applied and a set of Vector embeddings are retrieved. Popular similarity measures for ANN search include: Cosine Similarity, Euclidean Distance, Dot Product.

How are you using Vector DBs? Let me know in the comment section!

#LLM #AI #MachineLearning

-------

Be sure to ♻️ repost if you found the article useful and follow me and hit that bell if you want to get a daily dose of useful AI related content in your feed!

Here are some of my 𝗿𝗲𝗮𝗹𝗶𝘀𝘁𝗶𝗰 𝗽𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻𝘀 𝗳𝗼𝗿 𝗔𝗜 𝗶𝗻 𝟮𝟬𝟮𝟱!

𝟭. 𝗦𝗰𝗮𝗹𝗶𝗻𝗴 𝗹𝗮𝘄 𝗰𝗼𝗻𝘁𝗶𝗻𝘂𝗲𝘀 𝘁𝗼 𝗵𝗼𝗹𝗱 𝘀𝘁𝗿𝗼𝗻𝗴 in post-training and inference time compute: 

There are multiple aspects to this statement. 

➡️ We know that there is an enormous need for more compute to generate synthetic data needed to train reasoning models. 
➡️ However, there is also a need to pretrain larger models that can participate in evaluating these reasoning models and generate the synthetic data. 

✅ More low level details in the Newsletter that I will be releasing in January.

𝟮. 𝗜𝗻𝗳𝗶𝗻𝗶𝘁𝗲 𝗺𝗲𝗺𝗼𝗿𝘆 𝘄𝗶𝗹𝗹 𝗻𝗼𝘁 𝗯𝗲 𝗮𝗻 𝗶𝗻𝗵𝗲𝗿𝗲𝗻𝘁 𝗽𝗿𝗼𝗽𝗲𝗿𝘁𝘆 𝗼𝗳 𝘁𝗵𝗲 𝗟𝗟𝗠.

➡️ Infinite memory will be based on Long-term memory implementation principals where relevant context will be stored and retrieved from external systems.
➡️ There will be no infinite context window size.

𝟯. 𝗦𝗺𝗮𝗹𝗹 𝗺𝗼𝗱𝗲𝗹𝘀 will become standard in Agentic systems.

➡️ With enough tuning, small models can become great at planning specific tasks.
➡️ Generation latency is extremely important in complex agentic flows due to the amount of steps it could involve.
➡️ Price for the amount of tokens pushed through these systems could become a limiting factor for adoption if large models are used.

𝟰. New kind of 𝗔𝗜 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 𝗶𝗻 𝘁𝗵𝗲 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲 will slowly become a reality, but they will be far from mainstream by the end of 2025.

➡️ We live in a bubble, AI in Enterprise is hard due to less popular reasons:

👉 Security.
👉 Integration with internal systems.
👉 Accuracy requirements.
👉 Observability.
👉 …

✅ Many more AI Infrastructure startups will be spun up to help tackle this problem.

𝟱. Most 𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝗼𝘂𝘀 𝗔𝗴𝗲𝗻𝘁 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝘀 𝘄𝗶𝗹𝗹 𝗻𝗼𝘁 𝗯𝗲 𝗿𝗲𝗮𝗱𝘆 𝗳𝗼𝗿 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝗯𝘆 𝘁𝗵𝗲 𝗲𝗻𝗱 𝗼𝗳 𝟮𝟬𝟮𝟱.

➡️ We will continue working with Assistant based agents that augment our day to day work rather than take over jobs.
➡️ There will be marketing attempts to sell fully autonomous agents but they will fall flat in real world applications.

What are your predictions for 2025 when it comes to AI? Let me know in the comment section! 👇

#LLM #AI #MachineLearning

I love OpenAI’s UX. The release of 𝗖𝗵𝗮𝘁𝗚𝗣𝗧 𝗧𝗮𝘀𝗸𝘀 yesterday really shows how strong their 𝗣𝗿𝗼𝗱𝘂𝗰𝘁 𝗚𝗮𝗺𝗲 is, 𝗕𝗨𝗧 👇

The reactions to the release by AI Experts on the internet are INSANE and inadequate. While the UX improvement is significant, it is not a big technical innovation towards the Agentic world. Let me explain what I mean:

➡️ It is just a scheduler for your ChatGPT prompts.
➡️ It is most likely implemented by simply providing a scheduling tool to the ChatGPT agent.
➡️ People working in the industry have been doing this for the past 1-2 years already.

❗️ To show this, I plan to implement such capability from scratch in one of my newsletters next month.

✅ Having said that, it is inspiring to see how OpenAI is bringing scheduling capabilities via a text prompt to the masses.
✅ In general, it is obvious that OpenAI is winning the UX game in both consumer and developer markets.

Have you played with ChatGPT Tasks already? Let me know your opinion in the comment section 👇

#MachineLearning #AI #LLM

Taplio

Aurimas Griciūnas's Linkedin Analytics

What is Aurimas talking about?

Who is engaging with Aurimas

's Best Posts (last 30 days)

Want to drive more opportunities from LinkedIn?