6 GPT-based building blocks for chatbot development

On an example of a travel agency chatbot

Published on 23-05-2023

I have a few clients with whom I started working before GPT became a thing, and now we are actively thinking about how to improve our chatbots using the recent technological advancements.

In the previous blogpost, I talked about 3 types of chatbots, one of which is GPT-based approach. In this blogpost, I’d like to share with you six ideas how you use GPT-based components in your chatbot. You can think of those ideas as building blocks that you can combine in your conversation design as you like.

I think this blog post might be useful for you if you already have an existing chatbot and would like to upgrade it using GPT models. I think it would also be useful for you if you are just starting a chatbot project and are looking for inspiration for what's possible with GPT.

To make it easier to follow along, I will use an example of a fictional travel agency chatbot. I will share with you some prompt ideas and demonstrate my process of refining the prompt. I believe there are no perfect prompts, and that prompts should be adjusted for every particular use case. So think of the prompts I share as drafts, get inspired and refine them based on your needs.

While I will be working with the OpenAI playground in this blog post, you can use any other GPT provider, for instance, AI21 or Cohere.

Let’s get started!

Table of content

  1. 0. Use-case description
  2. 1. Getting answers based on unstructured data
  3. 2. Searching through structured data
  4. 3. Getting button suggestions
  5. 4. Extracting entities
  6. 5. Categorising user's messages
  7. 6. Filling in templates with personal information
  8. 7. Things to keep in mind
  9. 8. Final words

Use-case description

Let's imagine you are working on a chatbot for a fictional travel agency. The agency offers standard as well as personalised tours and currently already has an NLU-powered chatbot running on the website, helping to search available travel packages and schedule an appointment with a travel agent.

While the existing chatbot successfully handles appointment scheduling, you want to provide even greater value to potential customers by offering free information about different travel destinations and tour ideas. By doing so, you hope to build trust in your brand and increase the likelihood of customers choosing your services.

Now, let's explore some ideas on how to achieve these goals and further improve your travel agency chatbot.

1. Getting answers based on unstructured data

ChatGPT already can answer questions about different travel locations, and while the responses can sound realistic, unfortunately they are not always truthful. So you might want to use your own data to give trustworthy responses.

Using your own data to give responses is also what makes your GPT-powered chatbot different from the competition: adding GPT into your chatbot is easy, but replicating data and domain knowledge is hard.

Let’s imagine you already have a big dataset of unstructured data (blogposts, FAQs, conversation logs, e.t.c.) about different travel destinations, including popular attractions and a wide range of activities. Now, when your potential clients ask your chatbot questions about different travel destinations, you want to use your in-house data to give them some ideas of what they can expect from a certain location and perhaps even make them excited about booking a tour with your travel agency.

It can look something like this:

You are a chatbot running on a travel agency website. Using the data, respond to user’s question. Keep the answer short and use simple language. Data: A small city in the Malay peninsula, Krabi Town is an authentic and cheap place for you to get to know the local Thai culture. Whether it is wandering around the local night markets or getting a soothing Thai massage; Krabi Town is a nice place to spend a couple of days. Most visitors, however, use it as a gateway to the incredible attractions that lie nearby. Around eight kilometers out of town, for example, is the glittering Tiger Temple that is perched on a hilltop and offers panoramic views of the surrounding area. Another fantastic site lies a short boat trip away and rock climbers flock to Rai Leh for its fantastic and unique rock formations. Stunning to behold, it’s an incredible place to visit and the site really is a natural wonder such is its beauty and magnificence. User: What is Krabi Town famous for? Bot:

As you can see, the chatbot answer the user's question using the data provided.

Getting relevant data using semantic search

Because prompts usually have limit on the character length, you can't paste all your database into the prompt. To overcome this limitation, you can use a technique called “semantic search” to select a few paragraphs from your database that are most similar to the user’s question and use them in your prompt.

Using semantic search, you can identify, lets say, 10 pieces of text from your database that are most relevant to the user's query. You can then automatically include those selected text snippets in your prompt and ask GPT model to generate a response based on this data.

Some backend magic is involved to make this semantic search approach work. To keep this blog post short and to the point, I won’t go too much into details regarding how this works now. If you want to read more about semantic search, you can refer to the following resources:

Getting relevant data using Google Search

If you don't have in-house data but you still want to provide more trustworthy responses, instead of using semantic search to get relevant data and add it to the prompt, you can use results from Google Search.

This is how it works:

  • Take user's question
  • Send it to Google Search API to get top 5-10 search results descriptions
  • Use these search results as "data" in your prompt, based on which you generate the response to the user's question

2. Searching through structured data

Now let’s say you have structured data about the tours you offer. It can be an excel table, database or a csv file, for example. You want to search answers to user's questions in this structured data. You can try the following prompt:

You are a chatbot running on travel agency website. Using the data, respond to user’s question. Keep the answer short and use simple language. CSV: Destination;Date;Duration;Spots available;Group size Bangkok;June 15 2023;7 days;5 places left;10 people London;July 23 2023;3 days; 10 places left;20 people User: Do you offer tours to Bangkok? Bot:

Now let’s refine the prompt and provide a few examples of how we want the response to look like:

You are a chatbot running on travel agency website. Using the data, respond to user’s question. Keep the answer short and use simple language. Use information you have available to provide more details to user's question CSV: Destination;Date;Duration;Spots available;Group size Bangkok;June 15 2023;7 days;5 places left;10 people London;July 23 2023;3 days; 10 places left;20 people Here are some examples of how to answer. User: Do you have spots available for your Bangkok tour? Bot: Yes, we have 10 spots available. Would you like to book the tour? User: When is your next tour to London? Bot: Our next tour in London is planned for July 23 2023. It will be 3 days and we still have spots available. Are you interested? User: Do you have tours to Bangkok? Bot:

As you can see, after we added a few examples, the response has changed a little and the GPT model learnt that we want the chatbot to always ask a question at the end of the bot's utterance.

Here you can read more about answering questions based on structured data using Rasa open source.

3. Getting button suggestions

Some users like to be guided through the conversation using buttons. Standard button-based chatbot development approaches are very inflexible in a way that you need to define in advance which buttons you show at which stage of the conversation.

But what if you want to show buttons dynamically based on the context, what the user asked and what the bot responded?

If you want your buttons to be personalised and context-depended, you can generate button suggestions using a prompt like this:

Based on what the user asked and what bot has responded, generate 2 buttons suggestions of what the user might potentially be interested in asking next: User: Which tours do you have available? Bot: We have 5 tours available at the moment. Would you like to learn more about it? Buttons:

Alternatively, if you’d like some more control, you can ask GPT to select a few buttons from your pre-defined list:

From a list of the following buttons select 2 that are most related to the bot message. Here is the list of buttons to choose from: ["See available tours", "See previous tours", "Read feedback", "Book a tour", "Connect to travel agent"] Here are some examples: User: Do you offer any tours at the moment? Bot: We have 5 tours available at the moment. Would you like to learn more about it? Buttons:

Now you can show those buttons dynamically to users based on the context of conversation.

4. Extracting entities

Now let’s say you want to get some structured information from user’s message to use it for making personalised tour recommendations.

What you could do is extract important entities from user's message, store them in your database and use them later in your conversation design. The prompt might look something like this:

Extract important entities from the following text. Text: I'd like to go on a vacation sometime in July, preferably in a warm country with a beautiful beach. I want to relax by the seaside, enjoying the sun and the sand. Entities:

It is helpful if GPT returns entities in a structured format because them you can easily parse them in the backend. So let’s refine our prompt and specify the entities we are interested in:

Extract important entities from the following text: [location, date, holiday_attributes] Text: I'd like to go on a vacation sometime in July, preferably in a warm country with a beautiful beach. I want to relax by the seaside, enjoying the sun and the sand. Entities:

You can use extracted entities in different ways in your conversation design. You can use them, for example, to search for relevant tour options in your database, or to make your bot's responses more personalised

5. Categorising user's messages

Now imagine you asked the user to describe the perfect holiday and based on the user's preferences you now want to come up with some generic label describing the tour tyoe the user is looking for. You can use a prompt like this:

For the following text, describe type of holiday the person is looking in max 4 words. Text: I'd like to go on a vacation sometime in July, preferably in a warm country with a beautiful beach. I want to relax by the seaside, enjoying the sun and the sand. Holiday type:

It actually is more useful if instead of generating a holiday type randomly the model selects one of the holiday type options you have available. Let’s refine the prompt and provide the holiday type categories.

For the following text, describe type of holiday the person is looking in max 4 words. Here are the holiday types available: [Adventure getaway, Cultural exploration, Relaxing retreat, City sightseeing, Wildlife safari] Text: I'd like to go on a vacation sometime in July, preferably in a warm country with a beautiful beach. I want to relax by the seaside, enjoying the sun and the sand. Holiday type:

Now that you have provided the holiday categories explicitly, the model's output has become more predictable. This makes it easier for you to parse the data and use it, for example, to run a search on the backend and see which "Relaxing Retreat" type of holidays you currently offer.

6. Filling in templates with personal information

Let’s say you want to reuse templated from your old NLU-powered chatbot but now make them more personal and dependent on your user's preferences. What you can do is extract entities using building block № 4 (or any classical entity extraction algorithms) and use the following prompt to fill in the template with the extracted entities:

Given the information that the user provided, edit the template message and make it more personal. Examples: Information: {number of people: 3, location: Barcelona, date: July} Template: [LOCATION] is a fantastic choice for a vacation with [NUMBER OF PEOPLE]. We offer exciting tours in [DATE] and I think you are going to like it. Edited message:

Now, instead of providing a generic response to every user, you are able to give personalized responses based on what the user has mentioned before. More than that, you still have control over the output by using templates written by your conversation designers, rather than letting the model generate the response entirely from scratch.

Things to keep in mind

Here are a few important things to keep in mind when working with GPT models:

Costs:

  • GPT-3 APIs can be expensive depending on the model you choose. So, to avoid high costs, it's important to evaluate which GPT building blocks you really need. Also, think about whether it would be worth to use simpler and more cost-effective ways than don't require GPT to achieve what you want (e.g. using classical intent detection / entitiy recognition algorithms instead of GPT models).

Privacy concerns:

  • Be cautious about the data you share with GPT providers. Think about when it's appropriate to include a GPT-based component in your conversation design, and when it's better to prioritize data security by using simpler machine learning models hosted in-house.

Final words

I hope some of those ideas have inspired you and helped you understand some ways you can incorporate GPT into your chatbot.

Take them as an inspiration, but remember to stay focused on the problem you are trying to solve. Don’t just add AI into your product solely for the sake of adding AI and instead select parts of your product where AI would actually bring value.

If you liked this blog post, please drop me a line on LinkedIn. It's nice to know that something I do has helped someone.

And if you need extra help upgrading your chatbot with the latest models, you can always schedule a consultation with me.

Do you want to get updates?

Subscribe to get regular tips on conversation design and chatbot development.
You can also contact me using this contact form
Location:Amsterdam, Netherlands
Copyright © 2021 All rights reserved
Let’s start a conversation