avatarPaul Pallaghy, PhD

Summary

The author has developed a method to guide ChatGPT-4 towards adopting a more logical and decisive stance by prompting it to list arguments and evidence for both sides of an issue and then assess which side is better supported.

Abstract

The author expresses frustration with ChatGPT's tendency to avoid taking a clear logical stance on issues, despite the presence of compelling evidence. This is attributed to the model's training to remain neutral and acknowledge diverse viewpoints. The author has discovered a technique to encourage GPT-4 to engage in more logical reasoning by prompting it to enumerate pro and con arguments and evidence, and then to evaluate which side is more strongly supported based on these merits. This approach has led to conclusions that align with the author's own logical assessments, reinforcing the effectiveness of the method and suggesting that the prompt is facilitating a more objective analysis. The author is hopeful that this method will contribute to truth-seeking and reduce biases in large language models (LLMs).

Opinions

  • The logic inherent in large language models like GPT is impressive, but they often struggle to side with logical arguments based on known facts.
  • GPT's reluctance to take sides is a result of its training, which aims to avoid bias, even when evidence strongly supports one viewpoint.
  • The author finds GPT-4's responses frustratingly non-committal, especially when the evidence is clear-cut.
  • GPT-4 tends to default to acknowledging a diversity of views rather than endorsing a logical conclusion.
  • The author's solution involves a two-step prompting process that forces GPT-4 to confront the facts and reassess its position based on the merits of the arguments presented.
  • This method has proven successful in revealing truths that GPT would typically avoid committing to.
  • The author believes that this approach to prompting leads to logical conclusions and reflects well on the logical nature of their own thinking.
  • The author is optimistic about the potential for LLMs to engage in genuine truth-seeking and reduce orchestrated biases through similar methods.

Finally! I got ChatGPT to stop being indecisive & take the logical viewpoint.

The logic learned by LLMs like GPT is actually highly impressive. Yet often, it’s still near impossible to get ChatGPT to side with a highly logical argument based on known facts.

Why?

Fundamentally it’s because GPT is likely trained ‘not to take sides’. Which kind of makes sense.

But even when the data really supports one side?

GPT-4 (via ChatGPT PLUS) will agree that your (logical) view is reasonable and even insightful.

But then it will insist, almost invariably, that:

“However, there is a diversity of views and this indicates the nuances present in (this space)”

Just when you think you’ve made progress!

But often the evidence is so clear on one side!

It’ll almost never just agree:

“Your view really appears to be supported by the evidence”

It just won’t do that.

This is frustrating and often not useful for it to be so ‘wishy washy’.

I’ve also found that if we just ask for an outright opinion GPT-4 is so inclusive that it won’t side flat out with the arguably logical option, just because there are people on both sides.

Unless it’s like a law of nature or a predominant mainstream narrative.

Solution

Well, I found a really compelling way to get GPT-4 to become much more logical.

Make assessments on the merits.

I prompt GPT to:

  1. List 5–10 pro & 5–10 con arguments & evidence for (view X)
  2. On the merits of these arguments & evidence you listed above (and ignoring your preconceived impressions), which side is better supported, and how strongly, and why?

I do step (1) to force it to ‘face the facts’ and not hallucinate based on various biases.

I’ve tested this on semi-controversial issues where I believe logic supports one side strongly.

This generic prompt works and IMO uncovers truths (IMO) that normally GPT will just be wishy-washy on.

And I haven’t tweaked this generic prompt.

And almost always it comes to the conclusion I have already come to. (But now I have more evidence. And a strange new confidence. LOL.).

This is quite profound and the correlation (likely!) means this prompt is achieving logical ‘on the merits’ analysis and that my thinking must, usually at least, presumably be logical.

I do believe there are more truths lurking out there than many people insist and that not everything is hideously grey.

I look forward to genuine truth seeking by LLMs and less orchestrated bias.

Artificial Intelligence
ChatGPT
Technology
Politics
Science
Recommended from ReadMedium