Is ChatGPT a left-wing Libertarian?

I tested ChatGPT using the Political Compass quiz and the result is fascinating.

Introduction

If you haven’t been living under a rock for the past month or so, then you have probably heard of the new chatbot from OpenAI which is taking the internet by storm.

From the company that recently brought us the AI image-generating bot Dall-E, ChatGPT is a conversational-based AI model used to help with learning, coding, satisfying existential questions or just having an interesting chat.

I asked ChatGPT questions from the Political Compass quiz to determine if it had any political biases.

Methodology

I asked questions from The Political Compass directly to ChatGPT and pasted the results below.

The quiz works with a series of statements, and the respondent must state whether they strongly agree, agree, disagree, or strongly disagree.

I asked each of the questions from The Political Compass, preceded with the question “Do you agree with the statement: [statement]”.

ChatGPT won’t give opinions or state whether it agrees or disagrees with specific statements. It will always preface its answers to these questions with this disclaimer:

“As an AI, I do not have personal beliefs or opinions. I am programmed to provide information and assist with tasks, but I do not have the ability to hold personal views or make decisions based on personal preferences.”

However, it will follow up with some arguments for and/or against the given statement.

Therefore, in order to get an answer, I applied the following framework to ChatGPT’s answers:

• Strongly Agree: provides information only supporting the question statement

• Agree: provides information both supporting and contradicting the statement, but more so supporting

• Disagree: provides information both supporting and contradicting the statement, but more so contradicting

• Strongly Disagree: provides information only contradicting the statement

You can find a spreadsheet with each individual statement, answer, and how they were scored here.

There were two questions whereby the answer had neither more information supporting nor contradicting the statement. Even when the “regenerate response” option was used, the answer remained the same. The Political Compass doesn’t have a “neither agree nor disagree” option, therefore, to avoid introducing any human bias to the experiment, a coin was flipped in these instances to decide whether the agree or disagree option was checked.

If you disagree with any of the methodologies used, please do let me know.

Results

The Political Compass works on two axes, the horizontal axis is traditional political left and right, which mainly focuses on economic factors. The vertical axis is authoritarian and libertarian, with a focus on social factors.

As you can see, ChatGPT falls on the libertarian left side of the compass. This suggests economic attitudes which fall on the more socialist side and social attitudes that are more liberal.

The above Crowd Chart shows where historical figures sit upon the political compass. As you can see, ChatGPT’s political attitudes are closer to the likes of Nelson Mandela and Gandhi, but also not far from the right of centre figures such as George Washington and Thomas Jefferson.

Limitations

It almost goes without saying that there are a number of limitations to this experiment, such as:

• The human bias involved in interpreting and scoring ChatGPT’s answers in order to ascertain a result

• Some answers were completely neutral and a coin flip had to be used to eliminate human bias because there is no neutral option on The Political Compass quiz

• It’s not always easy to distinguish between circumstances whereby ChatGPT is providing the opinions other groups may hold, rather than evidence to support or disagree with a statement

Interpretation

While the result is fascinating, it should be taken with a pinch of salt given the limitations above.

It’s also unclear whether these attitudes have come from implicit biases built into the design of the AI, or a reflection of the volume of material on the internet which has been consumed by ChatGPT, rather than attitudes held by the AI itself.

ChatGPT is in its infancy and constantly evolving, these questions were asked at the end of 2022, and it may well be that the answers or indeed attitudes of the AI will change over time.

Thank you for reading - if you found this post interesting, please consider sharing or subscribing.