In this video, we introduce AWS Bedrock, which is a new service for working with large language models (LLMs).
We begin by introducing Bedrock, and demonstrating how it can be used to access models from a range of providers through the playgrounds and APIs.
Then, we demonstrate integrating it with knowledge bases and agents in order to customise the behaviour of the LLMs responses into your specific domain and business.
Video Transcript
This week I've been at AWS re:Invent and, as you would expect, there has been a lot of discussion around artificial intelligence and machine learning.
Much of that discussion and many of the new feature announcements relate to this new AWS service called Bedrock, which is their platform for accessing and working with large language models.
With that in mind, I thought it was worth spending some time to introduce Bedrock. So, I'm going to have a few slides introducing it and then a demo as to how it is actually used in practice. I also wanted to spend some time talking about knowledge bases, which is how you connect the large language models to your own proprietary business data and agents where you can build ChatBots which incorporate those knowledge bases. These can be used by your employees or your customers to interact with your business information through a chat textual interface.
So firstly, what is Bedrock? As I mentioned, it's a platform for accessing and working with large language models. By now, I'm sure everyone's familiar with ChatGPT by OpenAI, but as you may be aware, there are other large language models in the wild, such as Claude by Anthropic, which is one of the leading models competing with OpenAI. There's Stable Diffusion, Llama 2 series by Meta, which is an open source model, and Amazon also has some of their own models. So Bedrock basically allows us to get access to all of these models in one place. It has what we call a playground, which is a web interface where I can chat with these models and build my solutions and test them and build templates. But more importantly, it also has API access, so I can build applications which integrate with those models via Bedrock. It's provided as a fully managed and serverless solution, so there are no servers, there's nothing to configure, there are no clusters or anything like that. I can just send my queries to Bedrock and get the responses back from models.
Commercially, it's paid for based on consumption-based pricing, so depending on the number of tokens which you send to and receive from your models, there will be a payment per token. And there are many features to kind of take those foundational models and then also tailor them to your specific use cases. For instance, you can fine-tune the models, you can connect them to your own knowledge bases, and you can build ChatBots which have been customized to your organization.
So why would we do this? Obviously, what we could do is go and form a commercial relationship and access those models individually. But there are some advantages to accessing them via AWS and allowing them to broker those connections. Firstly, what we do is we get all of the models in one place and we can access them via a single API, giving us a kind of abstraction layer which makes it easier to change models as new ones come out in the future. And what some people do is also combine models to build an end-to-end solution. Maybe one is better for images and one is better for text and one is better for code. And what you want to do is pick and choose the best model. And Bedrock allows us to do that with that layer of indirection.
Next, if you're in an AWS environment, Bedrock is obviously going to be tightly integrated into that. So, for instance, your billing will be integrated into your AWS account, security controls, governance through IAM, and integration with other services like Lambda and databases, some of which we're going to see today. Next, Bedrock does add some of those additional features that we talked about. So the point-and-click connection to knowledge bases for RAG, which we will augment in generation, and the agent-based workflows which connect with Lambda. So Bedrock basically goes above and beyond what you'd get by working purely with the LLM providers.
And finally, there is a kind of reliability argument for working with AWS. As we know, they have a great track record in terms of reliability and security and compliance. So we can have a higher degree of confidence when we're working with AWS than we may do with working with a kind of new AI startup. So I'm now going to move into a demo to explain how this is used in practice. So here I'm logged into the AWS console on the welcome page, effectively for Bedrock. And the first thing I'll point out is Bedrock isn't available in all regions yet, and there are also differences. For instance, in Frankfurt, all of the models are not available yet. So I'm going to concentrate in US East 1, which seems to be the most fully-featured region for Bedrock.
Now, over in the Getting Started section, this basically allows us to navigate around Bedrock. So we have some training at the bottom here, links to the various playgrounds which we're going to look at in a moment, and starting points to go and read about the individual foundational models. If I click in the Provider section, this again is basically just information about what the model is, how it works, and how do we interact with it through APIs. And I can flick between the different model providers like AI21, Amazon, and Anthropic, Coher, Meta, and Stability API, which are our six providers as of today.
Within the examples section, we'll see that each of those six providers has a number of examples for interacting with it. They may include things like entity extraction, code generation, customer service, translation, summarization, stuff like that. So here I've clicked on a customer service FAQ ChatBot, and it's given us an example as to how we would interact with that bot optimally. And likewise, the Llama 2 debug code example has given us an example prompt and an example response. So think about that getting started section, it's just like information and documentation. Next up are the playgrounds, which is like an ability to interact with these models through the console. And there are separate playgrounds for text, chat, and images.
Firstly, if we do a text example, I'm going to select Anthropic, Claude 2, Version 2, and we're going to begin chatting with this through the playground. So I'm going to ask a random question, like tell me three interesting facts about Las Vegas, which is where I'm sitting now at the end of AWS re:Invent Conference. And sure enough, it's responded with three facts, with it's home to the largest hotel in the world. There are more hotel rooms in Vegas than in the entire state of Hawaii. And the Las Vegas strip is not actually located within the city of Las Vegas. Likewise, on the image playground, again, we have to select a model. So I'm going to select the Amazon Titan image model. And I'm going to make a random request of this through the playground, something like, can you give me a picture of a green cat in Las Vegas on the strip? So I'm going to press Run. And by default, it's going to give me three images. And you can see we can tweak that in the bottom right. And relatively quickly, it's given us three images which have been created by the Titan model. I can do things like add negative prompts, change the orientation and the size and request more or less images with each batch. So I'll do that, I'm going to say only give me one image. And you can add a negative prompt here. I'm going to say give me an image with no palm trees, which I think is how the negative prompts are used. After another moment, we will see that the images returned. And sure enough, the palm trees have been removed and we only have one image.
A second way of interacting with the playgrounds is via the examples. So we looked at some of the examples earlier. But what we can do is navigate to the examples area, choose one and then click the link in the top right and it will take us to the playground with the prompt already populated. So here we had a meeting transcript and we have asked for a list of action items for each person in the meeting. If we were to look at this code generation example, integrating with Claude, we'll click open in playground. The prompt has been populated and we can make adjustments to the request. So stuff like the temperature, which is a model of creativity. Now when I click run, that prompt is sent through the model. And in this case, the Python code, which we requested has come back. So this is basically just a different way of interacting with the playgrounds via the examples. And it's obviously a great way to learn and understand the capabilities of the ChatBots. The other thing I should point out is this model access section. So by default, we don't have access to all of the models. And you have to go and specifically request access to them. And we've done that for kind of role-based access control reasons. Perhaps you don't want all people to have access to all models. So they are putting that behind IAM so we have some controls. And there's also an end-user license agreement step. So you are also agreeing to terms by explicitly accessing the models in this way.
You can see that I've already requested access to several models. Now, I'll demonstrate the process of requesting access to a stable diffusion model. When I make the request, it shows as 'in progress.' These requests can take a minute or so to provision and become available.
Finally, access to the Stability AI model has been granted. I'll quickly test it to confirm its availability. In the examples route, you can search for a specific provider. I'll select the model I just requested access to.
Opening it in the playground, we request an image of a Sri Lankan tea plantation. A good image is returned. If I tweak the request, like asking for the same image but at night, we can see the image is created. It's more like dusk, but you can see it attempted to do the right thing.
Now, I'd like to talk about APIs. So far, we've been using the playground, which is useful for experimenting with these models, testing their behavior, and building templates. However, for integration into websites, applications, and mobile apps, we use APIs. Bedrock is fully API-enabled and accessible through various SDKs for languages like Python, JavaScript, Java, Go, etc. I'll demonstrate using the AWS CLI, but the concepts are similar across programming languages.
At the command line, 'AWS Bedrock help' provides commands for interacting with Bedrock. For example, 'AWS Bedrock list foundation models' returns data. Proper configuration of the AWS CLI command with access keys is necessary for this.
To run commands against models, use the AWS Bedrock runtime endpoint with the 'invoke model' command. Here, I'm requesting a story about two dogs, limited to less than 300 tokens, and the response is outputted to a file.
Switching models, for example from Claude to Meta Lama, and sending a similar prompt yields different stories. This illustrates using different models and combining them for an end-to-end user experience.
Bedrock also allows customization of models with business-specific information through knowledge bases. For illustration, I created a document: 'If monkeys are red, they're called David; if blue, Emma; if green, Benjamin.' This document is used to ask a ChatBot questions about this monkey population and test reverse reasoning.
The document, a PDF in this case, is uploaded to an S3 bucket. In Bedrock, I build a knowledge base based on this bucket. The knowledge base is named 'Monkey's Knowledge Base,' with a description related to monkeys' colors and names. The URI of the S3 bucket is pasted in, and AWS OpenSearch is used for text-to-vector conversion.
Once the OpenSearch database is ready and contains the text from the PDF, we can test interactions with the ChatBot. Questions like 'If monkeys are red, what are their names?' return accurate responses based on the PDF content.
The ability to customize large language models with proprietary information and interact in sophisticated ways based on our own business data is powerful and relatively easy to implement within Bedrock.
Finally, I introduced agents - ChatBots based on knowledge bases. These agents can guide users through steps and call external functions like registering issues or making bookings. For instance, I built an agent based on Claude v2, instructing it to behave like an angry person giving overly long and detailed answers.
In conclusion, Bedrock provides a platform for working with large language models. We've explored interacting with models, API integration, building and querying knowledge bases, and the concept of agents for guiding business processes. For those interested in integrating Bedrock into their business, I'm available for an informal conversation. This technology offers significant business value.