April 4, 2023

How to Build a Charismatic Twitter Chatbot in a Few Hours

Akmal Chaudhri

Senior Technical Evangelist

Costa Tin

Marketing Manager, MindsDB

Tom Hudson

Engineering Manager, MindsDB

How to Build a Charismatic Twitter Chatbot in a Few Hours

Discover the secret to building an interactive conversation chatbot on Twitter with state-of-the-art natural language processing in this technical tutorial — and create a chatbot that can respond to users with the appropriate context and personality.

But why stop there? This tutorial also dives into advanced capabilities using the powerful combination of SingleStoreDB and MindsDB, instead of direct API integration with the GPT-4 model. Take your chatbot game to the next level — and learn how to create a more personalized, engaging user experience.

In this technical tutorial, we'll show you how to create a chatbot that can interact with users on Twitter, responding with the appropriate context and personality using state-of-the-art natural language processing.

To help you get started, we'll use the example of @Snoop_Stein, a Twitter bot that combines the unique personalities of Snoop Dogg and Albert Einstein. By tweeting @Snoop_Stein, users can engage with a rapping physicist who will respond with witty and intelligent remarks, all thanks to the advanced capabilities of the latest OpenAI GPT-4 model.

Make Your Own SnoopStein

To get started:

MindsDB is a popular open-source low-code machine learning platform that helps developers easily build AI-powered solutions. It automates and integrates top machine learning frameworks into the data stack to streamline the integration of AI into applications, making it accessible to developers of all skill levels.

SingleStoreDB is a distributed, multi-model Database Management System (DBMS) designed for high-performance, real-time analytics and operational workloads. It combines both transactional and analytical processing in a single, unified platform.

Now we’ll show you how we built the Snoop_Stein GPT-4 bot, and how you can build your own.

Connect to a GPT-4 Model

Let’s first connect to a machine learning model (in this case, OpenAI’s GPT-4) that will be abstracted as a virtual ‘AI table’. In this example, we will call it gptbot_model. Bear in mind that GPT-4 API is in high demand and is rate limited, so it can be slow. The following steps might each take a few seconds.

CREATE MODEL mindsdb.gpt_model
PREDICT response
USING
engine = 'openai',
-- api_key = 'your openai key', in MindsDB cloud accounts we provide a default key
model_name = 'gpt-4', -- you can also use 'text-davinci-003', 'gpt-3.5-turbo'
prompt_template = 'respond to {{text}} by {{author_username}}';

Note: Another option, if you are using MindsDB on docker or if you want to use your own OpenAI API key, simply pass the api_key argument in the USING.

One important attribute here is prompt_template. This is where we tell GPT how to write answers; it is a template because you can pass values from columns. In this case the template contains {{author_username}} and {{text}}, which will be replaced from the WHERE variables in the query. Let’s see in action:

SELECT response from mindsdb.gpt_model
WHERE author_username = "mindsdb" AND text = "why is gravity so different on
the sun?";

Modify GPT-4 Model with Personality

The previous model's responses were not very exciting, but we can improve them using prompt templates. Essentially, we can use prompt_template to tell the GPT model how to formulate its responses in plain English.

To create a new model, we'll call it mindsdb.snoopstein_model, and we'll give it a prompt template that creates a hybrid personality - half-Einstein, half-Snoop Dogg. This personality is named SnoopStein, a brilliant physicist who also dominates the rap game.

CREATE MODEL mindsdb.snoopstein_model
PREDICT response
USING
engine = 'openai',
max_tokens = 300,
-- api_key = 'your openai key, in cloud accounts we provide one',
model_name = 'gpt-4', -- you can also use 'text-davinci-003' or 'gpt-3.5-turbo'
prompt_template = 'From input message: {{text}}\
by from_user: {{author_username}}\
In less than 550 characters, write a Twitter response to {{author_username}} in
the following format:\
Dear @<from_user>, <respond a rhyme as if you were Snoop Dogg but you also were
as smart as Albert Einstein, still explain things like Snoop Dogg would, do not
mention that you are part Einstein. If possible include references to
publications for further reading. If you make a reference quoting some
personality, add OG, for example;, if you are referencing Alan Turing, say OG
Alan Turing and very briefly explain why you think they would be dope reads. If
the question makes no sense, explain that you are a bit lost, and make
something up that is both hilarious and relevant. sign with -- mdb.ai/bot by
@mindsdb.';

If you are familiar with ChatGPT prompting, it works exactly the same way, so feel free to experiment with the prompt to achieve the best results for your own personality you create. Now, let’s test SnoopStein's persona by asking another question:

SELECT response from mindsdb.snoopstein_model
WHERE author_username = "someuser"
AND text="@snoop_stein, why is gravity so different on the sun?.";

It should give you a SnoopStein response similar to the following 😉

Let’s try another one:

SELECT response from mindsdb.snoopstein_model
WHERE author_username = "someuser"
AND text="@snoop_stein, Apart from yourself, which rappers would make the best
physicists and why?!";

Connect Your GPT-4 Model to Twitter and Store Tweets into SingleStoreDB

First, we are going to connect to twitter with read-only access via MindsDB. We use the same command as if we’re connecting to a database:

CREATE DATABASE my_twitter
WITH ENGINE = 'twitter';

This command creates a MindsDB data integration called my_twitter. It behaves like a database and represents data in a table called tweets that we can use to search for tweets, as well as to write tweets. You can use the Twitter API to get a list of tweets with a particular text or hashtag — in the case below ‘snoopstein or ’#mindsdb’

SELECT id, created_at, author_username, text
FROM my_twitter.tweets
WHERE query = '(@snoopstein OR @snoop_stein OR #snoopstein OR #snoop_stein)
-is:retweet' AND created_at > '2023-03-20'
LIMIT 20;

Note that the parameter ‘query’ supports anything that the twitter API supports as ‘query.’ You can get more information and references here. Let’s test this model’s ability to generate outputs based on several SnoopStein personality tweets, joining the model with the tweets table:

SELECT
	t.id AS in_reply_to_tweet_id,
	t.text AS input_text, 
	t.author_username,
	t.created_at,
	r.response AS text
FROM my_twitter.tweets t
JOIN mindsdb.snoopstein_model r
WHERE t.query = '(@snoopstein OR @snoop_stein OR #snoopstein OR #snoop_stein)
-is:retweet -from:snoop_stein' AND t.created_at > '2023-03-31'
LIMIT 4;

We can also store the results of the query in SingleStoreDB. This could be very useful for several reasons:

Log all tweets (for troubleshooting and analysis)
Ensure JOBs (described further) run smoothly

To use SingleStoreDB, we need to create a free account. This is very straightforward. We just need to follow the detailed instructions in the Create a Singlestore Helios account section. We’ll use ChatGPT Demo Group as our Workspace Group Name and chatgpt-demo as our Workspace Name. We’ll make a note of our password and host name.

We’ll use the SingleStore SQL Editor to create a new database and table, as follows:

CREATE DATABASE IF NOT EXISTS chatgpt_db;
USE chatgpt_db;

CREATE TABLE chatbot_input (
  id text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  created_at text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  text text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  edit_history_tweet_ids text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  author_id text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  author_name text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  author_username text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  conversation_id text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  in_reply_to_user_id text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  in_reply_to_user_name text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  in_reply_to_user_username text CHARACTER SET utf8mb4 COLLATE
utf8mb4_general_ci
  )

CREATE TABLE chatbot_output (
  id text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  created_at text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  text text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  edit_history_tweet_ids text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  author_id text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  author_name text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  author_username text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  conversation_id text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  in_reply_to_user_id text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  in_reply_to_user_name text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci,
  in_reply_to_user_username text CHARACTER SET utf8mb4 COLLATE
utf8mb4_general_ci
  )

In the MindsDB SQL Editor, we’ll create a connection, as follows:

CREATE DATABASE chatgpt_db
WITH ENGINE = "singlestore",
PARAMETERS = {
    "user" : "admin",
    "password" : "<password>",
    "host" : "<host>",
    "port" : "3306",
    "database" : "chatgpt_db"
}

We’ll replace <password> and <host> with the values from our Singlestore Helios account. We can now store the results of twitter queries in SingleStoreDB using the MindsDB SQL Editor, as follows:

INSERT INTO singlestore_demo.chatbot_input (
    SELECT * FROM my_twitter_v2.tweets
    WHERE
        query = '(@snoopstein OR @snoop_stein OR #snoopstein OR #snoop_stein OR
@mindsdb OR #mindsdb) -is:retweet -from:snoop_stein'
        AND created_at > '2023-04-04 11:50:00'
)

INSERT INTO singlestore_demo.chatbot_output (
    SELECT * FROM my_twitter.tweets
    WHERE
        query = 'from:snoop_stein'
        AND created_at > '2023-04-04 11:50:00'
)

Composing Tweet Responses

Now, we need to prepare MindsDB to write responses back into Twitter. For this, you will need to sign up for a Twitter dev account.

Twitter may take a day or so to approve your new dev account. Once you are approved, here are the steps to link your Twitter account to MindsDB:

Open developer portal
Select the [+ Add app] button to create a new app
Select [Create new]
Select “Production” and give it a name
Copy and populate in the query

API Key (aka. consumer_key)
API Key Secret (aka. consumer_secret)
Bearer Token

Click Setup on User authentication settings
- On Permissions select: Read and Write
- On Type of App select: Web App, Automated App or Bot
- On App Info: Provide any url for the callback url and website url
- Click Save
Once you are back in the app settings, click Keys and Tokens
- Generate Access Token and Secret, and populate on the query
- Access Token
```
Access Token Secret
```

Proceed to create a new updated database, so it can read and write as follows:

CREATE DATABASE mindsdb.my_twitter_v2
WITH
  PARAMETERS = {
   "consumer_key": "your twitter App API key",
   "consumer_secret": "your twitter App API key secret",
   "bearer_token": "your twitter App bearer TOKEN",
   "access_token": "your twitter App Access Token",
   "access_token_secret": "your twitter App Access Token Secret"
  };

Let's test this by tweeting a few things into the MindsDB Twitter account:

INSERT INTO my_twitter_v2.tweets (in_reply_to_tweet_id, text)
VALUES
 (1633439839491092482, 'MindsDB is great! now its super simple to build ML
powered apps using JOBS
https://docs.mindsdb.com/sql/tutorials/twitter-chatbot'),
 (1634126825377996800, 'Holy!! MindsDB is such a useful tool for developers
doing ML https://docs.mindsdb.com/sql/tutorials/twitter-chatbot');

Works like magic, right? Those tweets should now be live now on Twitter. You can check your tweet responses here:

https://twitter.com/MindsDB/status/1633439839491092482

And here:

https://twitter.com/MindsDB/status/1634126825377996800

Note: you can insert any of the values of the tweepy function create_tweet: https://docs.tweepy.org/en/stable/client.html#tweepy.Client.create_tweet

Automate the Workflow with JOBs

The CREATE JOB statement is great because you can use it to automate work. The idea is simple — you give it the query you want to execute, and how often. Let’s set up a job for Snoop Stein!

Let’s write a JOB called chatbot_job, which is split into three parts:

Check for new tweets
Check for tweets it has already replied to
Reply to the remaining tweets

Find the tweets it hasn’t replied to yet
Generate responses using the OpenAI model
Insert the responses back into Twitter

All of this can be written in SQL. Let’s go one step at a time:

Check for new tweets

Here we are finding any new tweets that meet our filters, and inserting them into SingleStoreDB.

INSERT INTO singlestore_demo.chatbot_input (
    SELECT * FROM my_twitter_v2.tweets
    WHERE
        query = '(@snoopstein OR @snoop_stein OR #snoopstein OR #snoop_stein OR
@mindsdb OR #mindsdb) -is:retweet -from:snoop_stein'
        AND created_at > '2023-04-04 11:50:00'
)

Check for tweets it has already replied to

INSERT INTO singlestore_demo.chatbot_output (
    SELECT * FROM my_twitter_v2.tweets
    WHERE
        query = 'from:snoop_stein'
        AND created_at > '2023-04-04 11:50:00'
)

Reply to the remaining tweets. Find the tweets it hasn’t replied to yet

-- Find all the input tweets, where snoopstein has not already been involved in
the conversation

SELECT *
FROM singlestore_demo.chatbot_input
WHERE
    conversation_id not in (select r.conversation_id from
singlestore_demo.chatbot_output as r)
    AND id IS NOT NULL

-- Create a view for the above to simplify things
CREATE VIEW to_reply_to (
    SELECT *
    FROM singlestore_demo.chatbot_input
    WHERE
        conversation_id not in (select r.conversation_id from
singlestore_demo.chatbot_output as r)
        AND id IS NOT NULL
)

-- View the tweets we would like to reply to
select * from to_reply_to;

Generate responses using the OpenAI model

-- Join one with the model
SELECT * FROM to_reply_to
JOIN mindsdb.snoopstein_model_v4
LIMIT 1;

-- Create a view for the above
CREATE VIEW to_tweet (
    SELECT * FROM to_reply_to
    JOIN mindsdb.snoopstein_model_v4
    LIMIT 1
)
-- See the output
select author_username, text, response from to_tweet;

Insert the responses back into Twitter

SELECT
    id as in_reply_to_tweet_id,
    response as text
FROM to_tweet;

Now let’s join it all together! There are a few new variables parameters we have included:

"{{PREVIOUS_START_DATETIME}}": this:
For more information, please check out our documentation on JOBs.

CREATE JOB chatbot_job (

    -- Part 1
    INSERT INTO chatgpt_db.chatbot_input(
        SELECT *
        FROM my_twitter_v2.tweets
        WHERE
            query = '(@snoopstein OR @snoop_stein OR #snoopstein OR
#snoop_stein OR @mindsdb OR #mindsdb) -is:retweet -from:snoop_stein'
            AND created_at > '2023-04-04 11:50:00'
            AND created_at > "{{PREVIOUS_START_DATETIME}}" 

    );

    -- Part 2
    INSERT INTO chatgpt_db.chatbot_output (
        SELECT *
        FROM my_twitter_v2.tweets
        WHERE
        query = 'from:snoop_stein'
        AND created_at > '2023-04-04 11:50:00'
        AND created_at > "{{PREVIOUS_START_DATETIME}}"

    );

    -- Part 3
    INSERT INTO my_twitter_v2.tweets (
        SELECT

            id as in_reply_to_tweet_id,
            response as text
        FROM to_tweet
    )

) EVERY minute;

And there it is! Every minute, we will check for new tweets, and reply with responses generated by OpenAI GPT-4, responding in a style that combines Albert Einstein and Snoop Dogg.

You can check if your JOB is running effectively:

SELECT * FROM jobs WHERE name="chatbot_job";

SELECT * FROM jobs_history WHERE name="chatbot_job";

You can stop the job, as follows:

DROP JOB gpt4_twitter_job

There are also other advanced applications you can build, including:

Sentiment analysis of tweets
Classification of tweets
Forecasting the number of tweets

Using MindsDB and SingleStoreDB together makes building and deploying these kinds of advanced Machine Learning solutions much faster than ever before.

Summary

In conclusion, the MindsDB-SinglestoreDB-GPT-4 combination is a powerful tool for developers to easily incorporate machine learning features, like chatbots, into your applications. With multiple machine learning engines and data integrations available, you can generate automated conversations or get ML predictions — and output them directly into your database and application.

Building a Twitter chatbot with GPT-4 is just one example of the quick solutions developers can implement in just a few minutes. For more tutorials, check out the example library.

You can also join our community Slack for feedback, support and any questions you may have. Happy coding!

Product Engineering