Using LLMs and ChatGPT in oTree experiments
EDIT: Joachim Gassen has recently released a useful Python package that allows you to use large language models (LLMs) as bots like ChatGPT in oTree experiments. He also released a GitHub repository showcasing some examples.
With the growing popularity of large language models (LLMs) and OpenAI continuing to update ChatGPT, many of us have been thinking about using this technology for experimental research. Since the use-case for such a feature is, in theory, is endless, I wanted to share some recent developments in the oTree community. All credit for this particular post on the use of ChatGPT in oTree should go to Clint McKenna, who put together an amazing oTree app that allows participants to chat with ChatGPT through OpenAI's API. Please find his original blog post here.
The idea is relatively straightforward. OpenAI has opened API access for quite a few months now. This allows researchers to use prompts to provide intial instructions to the API. For instance, we could instruct ChatGPT to play a certain character and even set the temperature, which alters creativity in its responses. Please see an example of such a prompt below:
This prompt creates a person living in Texas (including the local slang) who tried to direct the conversation to college basketball. Here is an example of a dialogue with them:
Clint McKenna created a simple oTree app that allows participants to speak to one of these "bots." Everyone can clone the app or copy the Github code into their local oTree repository. The app is currently set to randomly assign participants to one of two bots, which you can alter in the _init_.py file. You can also change the creativity of the bots (i.e., temperature ranges between 0 and 2) and the model (it currently uses gpt-3.5-turbo) in the app's constants.
While oTree is free and open source, ChatGPT's API tokens are not. You will need to sign up for an OpenAI account with a payment option. Once you have an API key, you can add it as an environment variable and this will be imported when working with the application. For hosting the server on Heroku, you can add it with this command (replacing with your key):
heroku config:add CHATGPT_KEY=sk-.....
You can also add ChatGPT’s API key as an environment in Windows for local testing. For production, you should also add the OpenAI package to your requirements.txt file:
openai==0.27.0
The chat data are saved in as participant variables in the app. However, there is also a custom export function to retrieve all chat data in long form under the "data" tab.
Final Note
The launch of OpenAI's API feature offers endless opportunities to integrate LLMs into experiments. oTree is the perfect platform because it is versatile and easily integrates API prompts. In this post, I shared an example of an oTree app circulating in the oTree community that uses OpenAI's API to create personalities and characters for participants to chat with. However, a simpler application would be to allow participants to communicate with or use ChatGPT during the experiment. I hope this post was useful and that it gives more exposure to Clint McKenna's wonderful oTree app specifically and oTree as a platform for experimental research more generally.
