6 GPT-based building blocks for chatbot development

I have a few clients with whom I started working before GPT became a thing, and now we are actively thinking about how to improve our chatbots using the recent technological advancements.

In the previous blogpost, I talked about 3 types of chatbots, one of which is GPT-based approach. In this blogpost, I’d like to share with you six ideas how you use GPT-based components in your chatbot. You can think of those ideas as building blocks that you can combine in your conversation design as you like.

I think this blog post might be useful for you if you already have an existing chatbot and would like to upgrade it using GPT models. I think it would also be useful for you if you are just starting a chatbot project and are looking for inspiration for what's possible with GPT.

To make it easier to follow along, I will use an example of a fictional travel agency chatbot. I will share with you some prompt ideas and demonstrate my process of refining the prompt. I believe there are no perfect prompts, and that prompts should be adjusted for every particular use case. So think of the prompts I share as drafts, get inspired and refine them based on your needs.

While I will be working with the OpenAI playground in this blog post, you can use any other GPT provider, for instance, AI21 or Cohere.

Let’s get started!

Table of content

1. 0. Use-case description
2. 1. Getting answers based on unstructured data
3. 2. Searching through structured data
4. 3. Getting button suggestions
5. 4. Extracting entities
6. 5. Categorising user's messages
7. 6. Filling in templates with personal information
8. 7. Things to keep in mind
9. 8. Final words

Use-case description

Let's imagine you are working on a chatbot for a fictional travel agency. The agency offers standard as well as personalised tours and currently already has an NLU-powered chatbot running on the website, helping to search available travel packages and schedule an appointment with a travel agent.

While the existing chatbot successfully handles appointment scheduling, you want to provide even greater value to potential customers by offering free information about different travel destinations and tour ideas. By doing so, you hope to build trust in your brand and increase the likelihood of customers choosing your services.

Now, let's explore some ideas on how to achieve these goals and further improve your travel agency chatbot.

1. Getting answers based on unstructured data

ChatGPT already can answer questions about different travel locations, and while the responses can sound realistic, unfortunately they are not always truthful. So you might want to use your own data to give trustworthy responses.

Using your own data to give responses is also what makes your GPT-powered chatbot different from the competition: adding GPT into your chatbot is easy, but replicating data and domain knowledge is hard.

Let’s imagine you already have a big dataset of unstructured data (blogposts, FAQs, conversation logs, e.t.c.) about different travel destinations, including popular attractions and a wide range of activities. Now, when your potential clients ask your chatbot questions about different travel destinations, you want to use your in-house data to give them some ideas of what they can expect from a certain location and perhaps even make them excited about booking a tour with your travel agency.

It can look something like this:

As you can see, the chatbot answer the user's question using the data provided.

Getting relevant data using semantic search

Because prompts usually have limit on the character length, you can't paste all your database into the prompt. To overcome this limitation, you can use a technique called “semantic search” to select a few paragraphs from your database that are most similar to the user’s question and use them in your prompt.

Using semantic search, you can identify, lets say, 10 pieces of text from your database that are most relevant to the user's query. You can then automatically include those selected text snippets in your prompt and ask GPT model to generate a response based on this data.

Some backend magic is involved to make this semantic search approach work. To keep this blog post short and to the point, I won’t go too much into details regarding how this works now. If you want to read more about semantic search, you can refer to the following resources:

• OpenAI cookbook
• Cohere cookbook
• Cohere docs

Getting relevant data using Google Search

If you don't have in-house data but you still want to provide more trustworthy responses, instead of using semantic search to get relevant data and add it to the prompt, you can use results from Google Search.

This is how it works:

• Take user's question
• Send it to Google Search API to get top 5-10 search results descriptions
• Use these search results as "data" in your prompt, based on which you generate the response to the user's question

2. Searching through structured data

Now let’s say you have structured data about the tours you offer. It can be an excel table, database or a csv file, for example. You want to search answers to user's questions in this structured data. You can try the following prompt:

Now let’s refine the prompt and provide a few examples of how we want the response to look like:

As you can see, after we added a few examples, the response has changed a little and the GPT model learnt that we want the chatbot to always ask a question at the end of the bot's utterance.

Here you can read more about answering questions based on structured data using Rasa open source.

3. Getting button suggestions

Some users like to be guided through the conversation using buttons. Standard button-based chatbot development approaches are very inflexible in a way that you need to define in advance which buttons you show at which stage of the conversation.

But what if you want to show buttons dynamically based on the context, what the user asked and what the bot responded?

If you want your buttons to be personalised and context-depended, you can generate button suggestions using a prompt like this:

Alternatively, if you’d like some more control, you can ask GPT to select a few buttons from your pre-defined list:

Now you can show those buttons dynamically to users based on the context of conversation.

4. Extracting entities

Now let’s say you want to get some structured information from user’s message to use it for making personalised tour recommendations.

What you could do is extract important entities from user's message, store them in your database and use them later in your conversation design. The prompt might look something like this:

It is helpful if GPT returns entities in a structured format because them you can easily parse them in the backend. So let’s refine our prompt and specify the entities we are interested in:

You can use extracted entities in different ways in your conversation design. You can use them, for example, to search for relevant tour options in your database, or to make your bot's responses more personalised

5. Categorising user's messages

Now imagine you asked the user to describe the perfect holiday and based on the user's preferences you now want to come up with some generic label describing the tour tyoe the user is looking for. You can use a prompt like this:

It actually is more useful if instead of generating a holiday type randomly the model selects one of the holiday type options you have available. Let’s refine the prompt and provide the holiday type categories.

Now that you have provided the holiday categories explicitly, the model's output has become more predictable. This makes it easier for you to parse the data and use it, for example, to run a search on the backend and see which "Relaxing Retreat" type of holidays you currently offer.

6. Filling in templates with personal information

Let’s say you want to reuse templated from your old NLU-powered chatbot but now make them more personal and dependent on your user's preferences. What you can do is extract entities using building block № 4 (or any classical entity extraction algorithms) and use the following prompt to fill in the template with the extracted entities:

Now, instead of providing a generic response to every user, you are able to give personalized responses based on what the user has mentioned before. More than that, you still have control over the output by using templates written by your conversation designers, rather than letting the model generate the response entirely from scratch.

Things to keep in mind

Here are a few important things to keep in mind when working with GPT models:

Costs:

• GPT-3 APIs can be expensive depending on the model you choose. So, to avoid high costs, it's important to evaluate which GPT building blocks you really need. Also, think about whether it would be worth to use simpler and more cost-effective ways than don't require GPT to achieve what you want (e.g. using classical intent detection / entitiy recognition algorithms instead of GPT models).

Privacy concerns:

• Be cautious about the data you share with GPT providers. Think about when it's appropriate to include a GPT-based component in your conversation design, and when it's better to prioritize data security by using simpler machine learning models hosted in-house.

Final words

I hope some of those ideas have inspired you and helped you understand some ways you can incorporate GPT into your chatbot.

Take them as an inspiration, but remember to stay focused on the problem you are trying to solve. Don’t just add AI into your product solely for the sake of adding AI and instead select parts of your product where AI would actually bring value.

If you liked this blog post, please drop me a line on LinkedIn. It's nice to know that something I do has helped someone.

And if you need extra help upgrading your chatbot with the latest models, you can always schedule a consultation with me.