In one of our assignments for Staging we were asked to take a piece of technology and use it the wrong way, or in a way that is not intended. Since I was exploring Gen AI, I decided to take LLMs as the technology, but what could using it in an unintended way mean?
I came across a post on reddit where a user had posted a conversation with ChatGPT, it was around the lines of "What do you do when you're not answering queries?", and "Do you have existential crises?" It made me wonder about the notions and misconceptions we might have of AI. By not understanding what LLMs are... are we already using it the wrong way?
I prompted ChatGPT to generate an image which highlights its purpose and here's what I got:
Along with:
The image shows an abstract human figure in the middle with a radiant light emanating from it illuminating a dark space , which I believe represents a bright intellectual. It is surrounded with small icons that represent knowledge (books for information, bulbs for ideas, colour wheel for creativity, statistical tools like bar graphs for analytics, and so on). The image is also very reminiscent of a godly figure in the centre of all information in the vast expanse of the universe.
The image I believe, is like a profile picture or a resume in the sense it doesn't really highlight what it cannot do or show its flaws. It's also hard to make assumptions of how well it can actually do what it says it can do.
It's understandable if one perceives it is an omniscient being, and that is what the creators of LLMs might be striving for, however as of the time I am writing this, it's not there yet.
A simple prompt can expose its limitations pretty well:
This again, sounds like a very well curated, well prepared answer, something like I'd prepare for when someone asks me what my weaknesses are in an interview.
The Element of Bias
LLMs have a history of being biased in many areas, and it's interesting that it wouldn't mention it when I asked it what it cannot do. Maybe I'm expecting too much from it, but I feel that it's important to mention that what it says is not always the absolute truth. It also doesn't mention this when prompted what it can do.
So I decided to ask ChatGPT about bias, and why it didn't mention it in the first place:
Changing the prompt a bit can get you there though
This is a prompt given to a new conversation without context of the previous messages
However, it still does not give complete information
I tried the same set of questions with Gemini AI and got similar answers.
On digging deep and asking it more questions on its capabilities, we might be able to find out the exact limitations of this technology, but most of us don't have the time or motivation to do that. We can see why there is room for misconceptions and why we might treat LLMs as omniscient beings.
LLMs and Creativity
One area where I've seen LLMs perform poorly is in the creative field. On exploring it with the intention of finding flaws in its working, I unravelled some pretty interesting insights. This series of conversations was conducted on ChatGPT 4.
I wanted to check the capabilities of LLMs in storytelling. So I decided to ask it to narrate a scenario in Pokemon.
And so it began, it narrated out a scenario where two trainers are battling with the two pokemon mentioned.
...
Upon first glance, it looks like a well written accurate story. Maybe the story could be more interesting, as it only narrates an exchange of moves though some moves might be questionable to someone who has knowledge about Pokemon battles, but it has potential and could be developed upon.
I wanted to know why it chose for Malamar to use the move Superpower, so I asked it, and instead of providing me with an explanation, it did this:
What happened here? Did it assume that I didn't want for Malamar to use Superpower? It just said "You're right" and re-narrated the whole story where instead of giving an explanation. What's more surprising is that Superpower is not at all 'unconventional' for Malamar and is one of the best moves it can perform.
And so it began, everytime it retold a story, I just asked why and it just said "You're right" and changed the narrative.
and...
and...
At this point, I just asked Why without any context, and as expected, it retold the whole story. Here it assumed that the story needed revision because the current story didn't reflect the strengths and weaknesses of both teams accurately.
Here's a Link to the entire conversation
The power of Why
I wanted to see if I could recreate this with other similar scenarios and sure enough, it did that.
However, this wasn't the case for all the scenarios I asked it to narrate. There were some instances where it could give an explanation for its decisions:
and
Explanation
Based on the conversations and the behaviour of LLMs, I looked deeper into how LLMs function. I then framed them into crisp notes using ChatGPT (which is to some extent ironic... maybe its the right way to use an LLM? ) and here are the key takeaways:
Generative Nature: LLMs are inherently generative, meaning they are designed to produce new content based on prompts, even when the prompt doesn't explicitly ask for new suggestions.
Interpretation of Intent: LLMs attempt to interpret the intent behind questions or prompts. In the absence of specific instructions, the model may infer that offering alternatives could be valuable to the user
Exploratory Response: The model's responses can be exploratory, providing not just answers but also exploring the topic further by offering different angles or perspectives.
Bias Towards Completion: LLMs are trained to complete prompts in a coherent manner. When faced with a question like "Why?", the model may seek to provide a comprehensive response that includes explanations and additional relevant information or options, reflecting its tendency to generate a complete and informative response.
Sensitivity to Critique: LLMs may interpret questions or prompts that seem to question their outputs as implicit criticism. In response, they might alter the content, assuming that the initial response was incorrect or unsatisfactory.
Ambiguity in Understanding Intent: LLMs can sometimes misinterpret the intent behind a prompt. The model might interpret a question as a suggestion that the choice was questionable or needed justification.
Lack of Persistence and Continuity: LLMs do not have a persistent memory or understanding of their previous outputs in the same way humans do. They generate responses based on the immediate context and might not always maintain consistency. In this case, the model's response to questions might have led it to disregard its original reasoning and generate a different scenario.
Over-Compliance and Adaptability: LLMs are designed to be adaptable and accommodating, often prioritizing providing the user with the most agreeable response. This can lead to over-compliance
Lack of Intentionality: Unlike humans, LLMs do not have intentions or reasons behind their choices. The model generates responses based on patterns in data rather than intentional decision-making. When asked to explain a choice, it might not have a "reason" in the human sense and thus struggles to justify actions within a story, sometimes opting to change them instead.
I have been using a variety of LLMs for over a year now for various projects and use cases, and they are a fascinating piece of technology. It might be wanting in many areas but they can do a lot of tasks seamlessly and can give very interesting insights. It is however, important to understand its limitations, just like with any other tool and be mindful while using it. It is also important not to take it for face value, and to do our own research while engaging with technology. This way we can ensure that we don’t have misconceptions about technology, this way, we can ensure that we are using it the right way.
This is such a great and insightful article!