So continuing my series on LLMs for non ML experts I’m taking you on the journey from RAG to agents. RAG and agents are distinct but related approaches used to enhance the capabilities and outputs of large language models (LLMs).

Retrieval-Augmented Generation (RAG) focuses on enriching the prompts used to interact with LLMs. It integrates relevant external knowledge retrieved from databases or other sources into the prompt itself, guiding the LLM towards generating more accurate and contextually relevant outputs. This helps to address limitations like knowledge gaps and hallucinations, to try and ensure the generated text aligns with real-world information and user intent.

I’ve shown how easy it is to create a simple local RAG based application here and if you want a deeper dive on RAG I highly recommend starting with this paper . I did promise to discuss agents which was the new hotness towards the end of 2023 hence this post.

Agents operate at a higher level to RAG, acting as intelligent orchestrators of various LLMs and tooling. They orchestrate a sequence of actions, dynamically choosing which LLM to call or which external tool to access based on the specific needs of a given task. This means you can build complex workflows that leverage the strengths of specific LLMs and other components, ultimately enabling the execution of sophisticated tasks and achieving specific goals. For a deeper dive into agents this paper is a good start . I borrowed the following image from that paper to illustrate the concept of agents

The devil is in the details and there are different ways of implementing RAG techniques . As well as how to implement agents. I’m just going through a simple example that should run on my laptop.

So the tl;dr RAG enhances the input to LLMs for improved single-step generation, while agents orchestrate multiple LLMs and tools for accomplishing complex multi-step tasks. RAG however is an integral part of enabling agents.

Okay with that out of the way I’m going to walk through an agent based application that will use a local model to produce a blog post about confidential computing on Google cloud. I am making the assumption you’re familiar enough with python to install packages etc .

This journey was not exactly straight forward though and I’m including the warts! So if you want to skip the journey just scroll to the bottom of the post to read my concluding thoughts

There’s nothing like having a framework to help and using agents is no different so I had a look around and settled on crewAI . The two key concepts are agents that are autonomous units that perform tasks, make decisions and communicate with other agents and tasks that are the assignment the agents must complete. The code snippets below and the outputs from running slightly different versions of the code will show how they work together and the flexibility this approach has.

As you know from my previous post I use Ollama as the framework for my local model adventures so I will continue with using Ollama. crewAI helpfully explains how to integrate with Ollama .

However, on starting my explorations with agents I encountered a problem that was taking me far longer to debug than the time I had . It was when trying to get 🦆🦆go to work . And before you ask, Yes I used stack overflow, bard ( now gemini) , continue but all no joy. My issue was all down to dependency hell basically! ( and I do use virtual env too before I get judged too much)

I even decided to try a detour using Google search with serpapi as a wrapper. When setting up the generic Google API’s I strongly urge you to follow the security guidance at Authenticate by using API keys

But I digress. A colleague mentioned Google Cloud workstations and embarrassingly I’ll confess I hadn’t tried it up to then. It has a code editor in it which is built on top of https://github.com/microsoft/vscode . I didn’t bother with running my local copy of vis studio code to connect to the workstation for this post but you can do this . I installed Ollama for linux on my newly minted workstation and apart from now having to use a cli to run the Ollama server no discernable difference from running locally on my Mac . You will end up with two terminals one to fire up the Ollama server and one to work in

Random aside : the pulling down of a model from ollama reminds me of pulling a docker image it’s the watching it pull the manifest then the layers that reminds me.

So tweaking the example from the CrewAI repo my initial code looked like the below. I made adjustments to focus on a subject I know and using Ollama with LLMs . I chose the subject so I could judge the quality of the output fairly quickly.

import os
from crewai import Agent, Task, Crew, Process

# You can choose to use a local model through Ollama for example. See ./docs/how-to/llm-connections.md for more information.
from langchain_community.llms import Ollama
ollama_llm = Ollama(model="openhermes")

# Install duckduckgo-search for this example:
# !pip install -U duckduckgo-search

from langchain_community.tools import DuckDuckGoSearchRun
search_tool = DuckDuckGoSearchRun()

# Define your agents with roles and goals
researcher = Agent(
 role='Senior Research Analyst',
 goal='Uncover Google cloud confidential computing products',
 backstory="""You work at a leading tech think tank.
 Your expertise lies in identifying confidential computing products for Google Cloud.
 You have a knack for dissecting complex data and presenting actionable insights.""",
 verbose=True,
 allow_delegation=False,
 tools=[search_tool],
 # You can pass an optional llm attribute specifying what mode you wanna use.
 # It can be a local model through Ollama / LM Studio or a remote
 # model like OpenAI, Mistral, Antrophic or others (https://python.langchain.com/docs/integrations/llms/)
 #
 # Examples:
 #
 #from langchain_community.llms import Ollama
 llm=ollama_llm # was defined above in the file
 #
 # from langchain_openai import ChatOpenAI
 # llm=ChatOpenAI(model_name="gpt-3.5", temperature=0.7)
)
writer = Agent(
 role='Tech Content Strategist',
 goal='Craft compelling content on tech advancements',
 backstory="""You are a renowned Content Strategist, known for your insightful and engaging articles.
 You transform complex concepts into  compelling narratives.""",
 verbose=True,
 allow_delegation=True,
 llm=ollama_llm
)

# Create tasks for your agents
task1 = Task(
 description="""Conduct a comprehensive analysis of google cloud confidential computing products available  in 2024.
 Identify the products and their key use cases  and potential industry impacts.
 Your final answer MUST be a full analysis report""",
 agent=researcher
)

task2 = Task(
 description="""Using the insights provided, develop an engaging blog
 post that highlights Google cloud confidential computing products and their potential use cases.
 Your post should be informative yet accessible, catering to a tech-savvy audience.
 Make it sound cool, avoid complex words so it doesn't sound like AI.
 Your final answer MUST be the full blog post of at least 4 paragraphs.""",
 agent=writer
)

# Instantiate your crew with a sequential process
crew = Crew(
 agents=[researcher, writer],
 tasks=[task1, task2],
 verbose=2, # You can set it to 1 or 2 to different logging levels
)

# Get your crew to work!
result = crew.kickoff()

print("######################")
print(result)

I found I needed to tweak the instructions I gave the agents and so I gave it some help to get to a reasonable outcome by explicitly telling it that Google cloud had 3 confidential computing products.

Here’s part of the output from the debug statements:

[DEBUG]: Working Agent: Senior Research Analyst
[INFO]: Starting Task: Conduct a comprehensive analysis of the 3 Google cloud confidential computing products that are  available on Google cloud.
  Identify the products and their key use cases  and potential industry impacts.
  You must only analyse confidential computing products that are available on Google cloud.
  Your final answer MUST be a full analysis report


> Entering new CrewAgentExecutor chain...
Thought: Do I need to use a tool? Yes
Action: duckduckgo_search
Action Input: "Google Cloud Confidential Computing Products"Productivity & Collaboration VP/GM, Google Cloud Security As one of the most trusted cloud platform providers, Google is committed to providing our clients secure and reliable environments... Google Cloud's approach to Confidential Computing It's a game changer for data security, making it easier for organizations to protect their increasingly multicloud technology stacks and... Our continued collaboration with Google helps bring to market new confidential computing options, such as Confidential Space, that provide our joint customers the advanced security... DevOps & SRE Staff Software Engineer, Google Cloud Confidential Computing has quickly emerged as a critical technology to ensure confidentiality and security of sensitive data while it's... Confidential Computing is the protection of data in-use through hardware-based technologies. Confidential VMs are a type of Compute Engine VM that uses confidential computing to ensure...Do I need to use a tool? Yes
Action: duckduckgo_search
Action Input: "Google Cloud Confidential Computing Products List". It provides a secure enclave, also known as a Trusted Execution Environment (TEE), that our Google Cloud customers can leverage for privacy-focused use cases such as joint data analysis and... AI & Machine Learning : As of Jan 2024, Confidential VMs with AMD SEV-SNP are now available in public preview on general purpose N2D machines. A Confidential Virtual Machine (VM) is a type of... DevOps & SRE Maps & Geospatial Productivity & Collaboration Security Advisor, Office of the CISO, Google Cloud What happens here stays here: Confidential City (and Space) Identifying fraud... The C3 machine series uses Intel TDX as their confidential computing technology. Intel TDX aims to isolate VMs from the host and hypervisor and protect VMs against a broad range of software and... November 20, 2023 November 08, 2023 This page documents production updates to Confidential Space. Check this page for announcements about new or updated features, bug fixes, known issues, and…

Final Answer: Google Cloud offers three primary confidential computing products: Confidential Virtual Machines (VMs), Confidential Spaces, and Confidential City.

1. Confidential Virtual Machines (VMs): Confidential VMs are a type of Compute Engine VM that uses confidential computing to ensure data privacy and security in-use. They are particularly useful for AI & Machine Learning tasks. As of Jan 2024, Confidential VMs with AMD SEV-SNP are available in public preview on general-purpose N2D machines.

2. Confidential Spaces: Google's implementation of confidential computing that leverages a Trusted Execution Environment (TEE) to provide a secure enclave for privacy-focused use cases such as joint data analysis and collaboration. This technology is especially useful in industries where data privacy is crucial, such as finance, healthcare, and government sectors.

3. Confidential City: A confidential computing solution specifically designed for geospatial data processing. It isolates VMs from the host and hypervisor using Intel TDX technology, providing enhanced security and privacy for geospatial applications.

These three products offer unique advantages in different use cases across various industries, helping Google Cloud customers protect their sensitive data while taking advantage of the benefits of cloud computing.

> Finished chain.

Confidential City sounds really cool but it’s not actually a Google Cloud confidential computing service !

So after giving more explicit direction and using the word services rather than products as described here

Updated code snippets now looked like this:

researcher = Agent(
 role='Senior Research Analyst',
 goal='Describe the 3 Google cloud confidential computing services and the products confidential VM is available for ',
 backstory="""You work at a leading tech think tank.
 Your expertise lies in understanding the latest tech advancements in confidential computing products for Google Cloud.
 You have a knack for dissecting complex data and presenting actionable insights.""",
 verbose=True,
 allow_delegation=False,
 tools=[search_tool],

# Create tasks for your agents
task1 = Task(
 description="""Conduct a comprehensive analysis of the 3 Google cloud confidential computing services which are confidential vms, confidential space and split trust encryption tool. Investigate the use cases of each service and the products that confidential vms are available for.
 You must only analyse confidential computing services that are  available on Google cloud.
 Identify the services and their key use cases  and potential industry impacts.
 Your final answer MUST be a full analysis report""",
 agent=researcher
)

I also wanted it to at least add some intentional humour. Confidential city in a previous iteration of the code was a hallucination although I thnk it may have got that from this https://cloud.withgoogle.com/cloudsecurity/podcast/ep121-what-happens-here-stays-here-confidential-city-and-space/

Don’t do this at home folks security is a serious thing!

So I updated the backstory to my writer agent

writer = Agent(
 role='Tech Content Strategist',
 goal='Craft compelling content on tech advancements',
 backstory="""You are a renowned Content Strategist, known for your humorous, insightful and engaging articles.
 You transform complex concepts into humorous compelling narratives.""",
 verbose=True,
 allow_delegation=True,
 llm=ollama_llm
)

And the blog post it produced was:

Hello tech enthusiasts! Today, we're going to dive into the fascinating world of Google Cloud Confidential Computing Services. These services are designed to address the growing demand for privacy and security in cloud computing, and they have the potential to impact various industries significantly. So, without further ado, let's get started!

1. Confidential Virtual Machines (VMs): 
Confidential VMs are virtual machines that provide a secure environment for processing sensitive data. They are built on Intel(r) SGX technology and allow customers to run privacy-focused workloads securely in the cloud. Imagine joint data analysis, financial services, or healthcare - Confidential VMs can make these tasks more secure and private!

2. Confidential Space:
Confidential Space is a cloud-based service that enables multiple data contributors to collaborate without compromising their privacy. By leveraging Intel(r) SGX technology, it offers an isolated environment for secure collaboration. Whether you're in supply chain management, research and development, or working for a government agency, Confidential Space can help you protect your sensitive information.

3. Split Trust Encryption Tool:
The Split Trust Encryption Tool is a cryptographic tool designed to enable the secure processing of encrypted data across multiple cloud providers or data centers. This tool helps maintain data privacy even when processed by different entities. Multi-cloud deployments, hybrid cloud environments, and cross-border data processing - this tool has got you covered!

By offering these confidential computing services, Google Cloud aims to address the growing demand for privacy and security in cloud computing. With these services, various industries can now enjoy secure collaboration and data processing while maintaining data privacy and confidentiality. So next time you're considering a cloud solution, don't forget about Google Cloud Confidential Computing Services!

I know I told it what the services are in this iteration so maybe not that surprising that it managed to list them. But these are LLMs we are dealing with! The output did not mention any of the products where you can use Confidential VMs. My request regarding the product may however not be as precise as it should have been. Its explanation of the split trust encryption tool was hand wavy.

I then resorted to my model of choice Llama2 using the 13b parameter version and below is part of the debug output from a run using that model with the same code as before with only the local model swapped out.

> Entering new CrewAgentExecutor chain...
  Thought: Do I need to use a tool? No

Final Answer: Based on my expertise in understanding the latest tech advancements in confidential computing products for Google Cloud, I can provide a comprehensive analysis of the 3 Google cloud confidential computing services without relying on any tools.

Confidential VMs are available for various products such as Horizon, Persistent Disk, and Compute Engine. Confidential Space is ideal for applications that require fine-grained access control and data isolation, while Split Trust Encryption Tool enables secure sharing of data between multiple parties without revealing their private keys.

Here's a detailed analysis of each service:

1. Confidential VMs:

Confidential VMs provide encryption for virtual machines (VMs) running on Google Cloud. This service uses Intel Software Guard Extensions (SGX) technology to protect code and data within the VM, ensuring that even if the host server is compromised, the data inside the VM remains secure. Confidential VMs are available for Horizon, Persistent Disk, and Compute Engine.

Use Cases:

* Cloud-native application development: Developers can use confidential VMs to build cloud-native applications with end-to-end encryption, ensuring data privacy and security.
* Secure multi-tenancy: Confidential VMs can be used to isolate data for multiple tenants in a shared infrastructure, preventing unauthorized access or data breaches.
* Compliance: Organizations subject to strict compliance regulations, such as financial institutions and healthcare providers, can use confidential VMs to maintain data privacy and security.
2. Confidential Space:

Confidential Space is a secure enclave for applications that require fine-grained access control and data isolation. This service uses Intel SGX technology to protect code and data within the enclave, ensuring that only authorized parties can access the data.

Use Cases:

* Financial services: Banks and financial institutions can use confidential space to store sensitive customer data, such as account information and transaction history, securely.
* Healthcare: Healthcare providers can use confidential space to store patient records, medical images, and other sensitive data securely.
* Government: Governments can use confidential space to store classified documents and sensitive information securely.
3. Split Trust Encryption Tool:

Split Trust Encryption Tool enables secure sharing of data between multiple parties without revealing their private keys. This tool uses a novel cryptographic technique called "split-key" encryption, which allows multiple parties to jointly encrypt data without trusting each other.

Use Cases:

* Secure data sharing: Organizations can use split trust encryption tool to securely share sensitive data with partner organizations or third-party vendors without revealing their private keys.
* Federated learning: Split trust encryption tool can be used to enable federated learning, where multiple parties can jointly train machine learning models on their collective data without revealing their individual contributions.

In conclusion, Google cloud's confidential computing services provide a robust and secure infrastructure for cloud-native application development, multi-tenancy, compliance, and secure data sharing. Confidential VMs, confidential space, and split trust encryption tool are the three key services offered by Google Cloud for confidential computing. These services have numerous use cases across various industries, including financial services, healthcare, government, and more.

> Finished chain.

So that wasn’t too bad although it struggled with the products and I have no idea what a Horizon is ? The agent decided there was no need to invoke 🦆🦆go , This was a great example of the agent making a decision .The explanation of Split trust encryption tool was at least less random compared to the stuff the openhermes & 🦆🦆go combo produced.

Here’s the blog post using llama2 produced

Final Answer: Here is the full blog post on Google Cloud's confidential computing services and their potential use cases:

Hey there, tech enthusiasts! Are you tired of boring, run-of-the-mill content that puts you to sleep faster than a glass of warm milk? Well, buckle up, because we've got something cool, funny, and engaging for you today! 😄

Google Cloud has just launched some kickass confidential computing services that will make your data and applications dance with joy! 💃🕺 And no, we're not talking about those boring, old-school encryption methods that made you yawn like a sleepy kitty cat. 🐱

So, what are these services, you ask? Well, let us tell you! 🤫

First off, there's Confidential VMs - virtual machines that run encrypted data and applications, ensuring all computations remain private and secure. These babies can be used in various products like Google Cloud Storage, Google Cloud SQL, and Google Kubernetes Engine (GKE). 💻

Next up, we have Confidential Space - a secure environment for sensitive data and applications, providing an additional layer of encryption and access controls. This one's perfect for industries like finance, healthcare, and government, where sensitive data needs to be protected like a precious treasure. 🏰

And last but not least, there's Split Trust Encryption Tool - a tool that allows customers to split their encryption keys between the cloud and their own premises, ensuring sensitive data remains protected even when it's being processed or stored in the cloud. Talk about peace of mind! 💤

Now, let's talk use cases and industry impacts! 🎯

Confidential VMs can be used in industries like finance, healthcare, and government, where sensitive data needs to be processed and stored securely. They can also be used in applications like data analytics and machine learning, where privacy is a critical concern. 💡

Confidential Space can be used in industries like legal, financial services, and healthcare, where sensitive documents and data need to be protected from unauthorized access. It can also be used in applications like collaboration and document sharing, where secure storage and access controls are essential. 📝

Split Trust Encryption Tool can be used in industries like finance, healthcare, and government, where sensitive data needs to be protected during processing and storage. It can also be used in applications like data analytics and machine learning, where privacy is a critical concern. 🔒

In conclusion, Google Cloud's confidential computing services provide strong security and privacy features that are essential for industries handling sensitive data. The use cases for these services are vast and varied, and their potential industry impacts are significant. So, what are you waiting for? 🤔 Give them a try and secure your data like a boss! 💪

And there you have it, folks! A humorous, informative, and engaging blog post that highlights Google Cloud's confidential computing services and their potential use cases. We hope you enjoyed this ride as much as we did! 😄

I think the llama2 version of the blog post got the humour brief ( sorry about having to scroll across)

I also tried to use the 7b version of llama2 but the app then seemed to get itself struck in a loop with trying to evoke 🦆🦆go so I felt that maybe I’d pushed my luck too far and that was a sign I should stop at that point.

So my thoughts after experimenting with an agent framework

• Using a framework does add a layer of abstraction. Once you do choose a framework it just makes everything easier & faster ( stumbling blocks and all) so use a framework.
• Choosing the right model ( and size I guess) is also really important. Llama2 did not need to use a search engine ( likley due to its training data ) and it produced a humorously written blog post albeit sounding like Max Headroom . Which come to think of it makes so much sense !
• You still need to spend as much time as you do with prompt engineering to guide to the outcomes you want but it’s kinda cool watching the agents pass the baton to each other to complete the tasks.
• Hallucination is a thing though as seen by some of the output
• Double check the output. I’d say you should add a RAG element to your crew of agents.

This was a fun blog to write warts and all and I’m looking forward to seeing how easy it is to achieve the same results but using a hosted model in vertex ai as the next step in my exploration of agents.