Every technology news outlet is showcasing ChatGPT at the moment, because it can generate convincing human-like responses to simple text prompts. But how does ChatGPT actually work, and is it really intelligent?
Defining Artificial Intelligence
Artificial Intelligence is an umbrella term for a collection of allied disciplines within software engineering. Within those disciplines there are further sub-divisions, championing different methodologies or technologies. With AI, it feels like you can go on peeling layers from the onion for ever, hopelessly searching for a neat definition of what AI actually is.
Let’s try to come up with our own. “Artificial” is easy enough to define. It means something that was created to have the same properties or purpose as something that occurs naturally. An artificial lung is designed to do the job of its organic counterpart, so it warrants the label “artificial”.
Defining “intelligence” is much harder. Depending on who you ask, you’ll get a different answer. Imagine a room of philosophers, theologians, primatologists, psychologists and lawyers trying to define intelligence. The debate would never end. And that’s the problem.
How can we define Artificial Intelligence if we can’t even define intelligence? We’ll just have to side-step that shortcoming. Our working definition will be “If a digital system can perform some task that would ordinarily require human intelligence, then that system has artificial intelligence”.
That’s exactly what ChatGPT does. It interprets the input, and creates an appropriate response. Normally, that would require human intelligence. So ChatGPT must possess artificial intelligence. But that doesn’t make ChatGPT truly intelligent. It means it has the ability to mimic intelligence. ChatGPT is just following its programming. Very clever, very sophisticated programming, but programming nonetheless.
This Isn’t Hollywood
The science fiction trope of Artificial Intelligence is one of a self-aware, rational, general intelligence, with its own interests and emotions. It is aware of its own consciousness, and it knows that other entities such as people are conscious too. In Artificial Intelligence circle, this is called the Theory of Mind.
ChatGPT is nothing like that. Nor is anything else. We don’t have the ability to create anything remotely close to so-called “strong AI”. So what is going on inside ChatGPT, under the hood? How is the mimicry performed? One way to get an understanding of what a complex system like ChatGPT is doing internally, is to examine a simplistic version.
Eliza and Her Descendants
Computer programs that simulate conversations have been around for a very long time. One example that caused a stir at the time of its creation, was Eliza, created in 1964 by Joseph Weizenbaum. We wrote our own version of a modern Eliza, with an extensible vocabulary and an improved analysis of the user’s input. But it’s still a very simple program.
Here’s a portion of a conversation with our software.
> HELLO. I WILL TALK ABOUT WHATEVER SUBJECT YOU LIKE. PLEASE BEGIN.
< Hi. What is the date?
> THE DATE TODAY IS 03-FEBRUARY-2023.
< Wow. A long time until pay day.
> WHAT DO YOU DO TO EARN YOUR MONEY?
< Cyber security, data protection, and compliance.
> DO YOU THINK YOU SHOULD RECEIVE MORE MONEY FOR WHAT YOU DO?
< It's not that, I need to curb my spending!
> HOW COME YOU NEED TO CURB YOUR SPENDING?
< I'm a little ... frivolous with money.
> ARE RICH PEOPLE'S LIVES REALLY TROUBLE FREE?
< Probably not. But I'd say poor people's lives are more stressful.
> WHAT ABOUT THE PREPOSITION THAT MONEY IS THE ROOT OF ALL EVIL?
< Money is neutral. It can facilitate good or bad things.
> WHY IS MONEY IMPORTANT TO YOU?
< We all must eat!
> IF YOU COULD EAT WHATEVER YOU WANTED WHAT WOULD YOU CHOOSE?
That looks like the software has a grasp of the conversational thread, as it moves from topic to topic, and it has some understanding of implication and nuance. It knows that “pay day” is related to “money”, for example. On the face of it, that’s pretty impressive. But actually, no it’s not. That’s not what’s happening at all.
The software knows a lot of trigger phrases and keywords. If it spots any of those in the user’s input it assumes that’s the topic of the sentence. It has a set of canned responses for each topic. “Pay” is one of the trigger words for the “money” topic, so a response from the money group is used. Very simple, with no genuine understanding or intelligence required.
Sometimes responses from our program are written verbatim. Sometimes a portion of the user’s input is repeated back to the user. When the user says:
< It's not that, I need to curb my spending!
The software replies with:
> HOW COME YOU NEED TO CURB YOUR SPENDING?
It’s taken “curb my spending” and performed a viewpoint shift, changing “my spending” to “your spending”. These are examples of a few of its bag of tricks. Combined, they can give a convincing performance of a real conversation. In its internal traces we can see the topics it used and the phrases that triggered them.
> [038] 'TALKING ABOUT YOUR WAGES', 'PAY', 'WOW. A LONG TIME UNTIL PAY DAY.'
> [065] 'SAYING 'YOU NEED TO' SOMETHING OR OTHER', 'I NEED TO', 'IT'S NOT THAT, I NEED TO CURB MY SPENDING!'
> [037] 'TALKING ABOUT MONEY AND WEALTH', 'MONEY', 'I'M A LITTLE ... FRIVOLOUS WITH MONEY'
> [037] 'TALKING ABOUT MONEY AND WEALTH', 'POOR', 'PROBABLY NOT. BUT I'D SAY POOR PEOPLE'S LIVES ARE MORE STRESSFUL.'
> [037] 'TALKING ABOUT MONEY AND WEALTH', 'MONEY', 'MONEY IS NEUTRAL. IT CAN FACILITATE GOOD OR BAD THINGS.'
> [041] 'TALKING ABOUT EATING', 'EAT', 'WE ALL MUST EAT!'
ChatGPT: The Same, But Bigger and Better
ChatGPT is like this, but on steroids. And brain pills. And it swallowed an encyclopedia. Its input parsing and language analysis is incredibly sophisticated. It’s not limited to looking for simple trigger phrases, and it can extract many topics from a single user input. It also handles deeper linguistic constructs such as nuance, hint, and implication.
It also knows what is possible in our world. Our simple program dumbly goes along with anything:
< I have six knees in each leg.
> DO YOUR FRIENDS KNOW THAT YOU HAVE SIX KNEES IN EACH LEG?
ChatGPT knows what it has been told is untrue:
- I have six knees in each leg.
- I'm sorry but humans typically only have one knee joint in each leg. Having multiple knee joints is not a characteristic of the human anatomy.
ChatGPT’s set of responses is practically unbounded. ChatGPT uses machine learning. This means it becomes better at its nominated task the more it is used, the more it is trained and guided, and the more data it has to work with. It has no shortage of data. It was trained on billions of words and millions of articles from all over the internet.
ChatGPT is a stunning example of cutting edge AI and knowledge engineering. But remember, ChatGPT isn’t intelligent. It’s mimicking intelligence.
At the end of the day, it’s just following its programming.