Why an AI Pioneer Is Worried

00:05
Speaker 1
This year, one thing that a lot of people have been talking about is AI. Artificial intelligence.

00:12
Speaker 2
All right. Artificial intelligence has been the story of 2023, right?

00:17
Speaker 1
I had Chad GPT write my graduation speech.

00:20
Speaker 2
Groundbreaking year for artificial intelligence.

00:23
Speaker 1
Today, I let Chat GPT control an entire college day of my life. And honestly, I was not expecting to be this. And because so many of us are new to AI, we wanted to call up someone who's been thinking about it for decades. And you are referred to as one of the godfathers of AI. Is this a title that you like, that you use?

00:47
Speaker 2
No.

00:49
Speaker 1
That'S Yashua Bengio, a professor at the University of Montreal. He's considered one of the godfathers of AI because his work helped lay the foundation for many of the AI models in use today. Why did you get into AI?

01:07
Speaker 2
Intelligence has always sounded like so incredible and mysterious and important because it's what's special about us and what allows us to do great things. Understanding that and building computers that would be intelligent. That was so exciting. Still is, but now I'm concerned.

01:33
Speaker 1
Yashua has spent his entire career working to advance AI technology. Now he thinks the technology is moving.

01:42
Speaker 2
Too fast and just saying, oh, everything is going to be fine. And AI is useful. Yeah, it could be extremely useful. But the more powerful it is, the more useful it can be and the more dangerous it can be. It just goes together. It's hand in hand.

02:01
Speaker 1
Welcome to the journal, our show about money, business and power. I'm Kate Leinbaugh. It's Tuesday, December 19. You coming up on the show, one of AI's godfathers on why he's worried about the technology helped create. Yashua Bengio has long been interested in the relationship between humans and computers. And we heard that you read a lot of Sci-Fi as a kid, is that right?

02:49
Speaker 2
That's true, yes.

02:51
Speaker 1
What kind of SCi-Fi well, all the old stuff.

02:53
Speaker 2
I was a kid in the early 80s.

02:57
Speaker 1
Okay. Did that have anything to do with you getting into AI?

03:02
Speaker 2
Yeah, probably so. For example, I read the whole series of novels by Asimov on the irobot and company, all the Bradbury novels and so on. AI is very prominent in science fiction, so it was sort of a fantasy when I was a kid. But then when I studied computers at university and started to read real scientific papers about AI, I realized that maybe it was something actually possible.

03:37
Speaker 1
For a lot of AI researchers, the goal has been to create an intelligent computer where the computer would accomplish the same intellectual tasks that humans do. And Yasha's theory for how to do this was inspired by how the human brain processes information. This approach, known as deep learning, was on the fringes of AI research for a long time. But Yashua believed that understanding human intelligence was the key to building a machine that could learn.

04:09
Speaker 2
We are born with some knowledge that's wired in from our genome, but most of what we have in our brain comes from our own experience. And so there's a lot that we could learn about how humans learn, how animals learn. In order to design this new approach.

04:25
Speaker 1
To artificial intelligence, how would you build such a machine? A machine that could learn?

04:35
Speaker 2
You simply get better at anticipating what will come. And, of course, in order to do that, you need to build, implicitly an understanding of how everything you're seeing is related to each other. That's how we build an understanding of our physical environment, our social environment, our job, the games we play. So think about the games we play. Initially, we don't know how to play. Maybe somebody tells us the rules, but that's not enough to be good. And then we practice, and each time we play, we get a bit better because we see what worked and we see what didn't work.

05:13
Speaker 1
So if you wanted to teach a computer what a cat is, what it looks like, how could that work?

05:22
Speaker 2
You just show it lots of cat images, and it would try to kind of anticipate what cat images look like. So, for example, given an image, it could tell you if it's a cat or not. Does it look like the other cat images, or it doesn't. That's it. I'm simplifying. But that's basically how they learn what cats look like. And, of course, you could do it in parallel for cats, dogs, and tens of thousands of other categories. And now it recognizes objects.

05:55
Speaker 1
And this approach of deep learning showed a lot of potential.

06:00
Speaker 2
So we started making progress on these classical AI tasks and starting to beat other methods pretty significantly. It happened gradually in the 2000s, but what happened after 2010, 2012, more precisely, is that companies starting seeing this as something that could be very profitable. And they started hiring some of us and buying startups, starting to work on this and so on. So it became something industrial, like money is involved. And it's not just some crazy sort of scientist having fun, trying to figure out intelligence. It was something that could change the world.

06:44
Speaker 1
What did that feel like to you?

06:47
Speaker 2
Well, initially, I was quite happy. It was like, oh, that's confirmation that we are on the right path. People have been saying that the approach were following didn't go anywhere. We had trouble getting accepted in the broader community, but suddenly companies were very quickly developing products, and suddenly it was working a lot better.

07:10
Speaker 1
So you were vindicated?

07:12
Speaker 2
It was great. And my students could get jobs too, right? Well paid ones, too.

07:20
Speaker 1
Tech companies ran with deep learning for new AI systems. Some of these systems were so called language models. Instead of being trained on, say, cat pictures, they were trained on text. Chat GPT is based on a language model. It uses deep learning to essentially predict the next word in a sequence. This is how the chatbot answers questions, writes code, and does math. When Chat GPT was released, yashua's expectations were pretty low.

07:54
Speaker 2
I played a bit with it, and I was trying to nail it, like to find the cases where it would give wrong answers. They would have a snake, or it wouldn't be able to do some really simple task. I found that it would be very bad at doing simple arithmetic that a twelve year old could do easily.

08:11
Speaker 1
Do you remember exactly what you asked it?

08:14
Speaker 2
Yeah, like adding two or three digit numbers. So initially I thought, well, yeah, we're not there, and I still think we're not there. But after a few months of doing this, I realized, but wait, it knows so much stuff. No one knows as much as this machine does.

08:41
Speaker 1
The thing was, even though Chat GPT was making mistakes, yahshua saw parallels with how we humans think. He says the machine was showing something like intuition.

08:53
Speaker 2
Humans also make these mistakes if they don't take the time to think through carefully. So if I ask you to add two three digit numbers, and I ask you to do it really quick, so you don't have time to go through all the usual steps, you're going to give me, like, a rough thing, and it's going to be ballpark reasonable. And that's exactly what you get from Chat GPT. So it's much more like we are. And that's also why it fails, because it only relies on its intuition for everything. It just has a huge set of things on which it has intuition.

09:32
Speaker 1
So you kind of started to see some of yourself, like, your humanness in it, in a way, of course. What did that feel like?

09:42
Speaker 2
Well, in a way, it's not very surprising, because we have been designing these systems based on inspiration from human intelligence. But at the same time, it's really important for people to understand that this is an alien intelligence. Even though it may have some similarities with our intelligence, it is not human intelligence. It will probably never be. It might imitate us.

10:10
Speaker 1
Chat GPT just predicts the next word, right?

10:13
Speaker 2
The probability of the next word, and then it samples one of the possible words according to those probabilities more precisely. So what I mean is that each time you call it, you might get a different answer because it's really like a random thing. Just like if you ask me the same questions, I never answer exactly the same way. It's exactly the same thing for Chat.

10:32
Speaker 1
GPT, but that still feels a long way off from human level AI.

10:39
Speaker 2
No, I know people say this, but they are wrong.

10:43
Speaker 1
Tell me why.

10:45
Speaker 2
Well, very simple. If you try to predict the next word that I will say, there's like 100,000 words that I could say, roughly.

10:54
Speaker 1
I was going to say a million.

10:55
Speaker 2
Yeah, if you include rare words and proper nouns. So which of these 1 million words will I say? It's very hard to predict unless you understand what I'm talking about. Let's say I ask, I consider a sequence of words where it starts with the question, and then there's question mark, and then there is the answer. The answer is just the next word. So in order to provide that next word correctly, you need to understand the question and what it's about.

11:25
Speaker 1
There needs to be a level of understanding.

11:29
Speaker 2
There is a level of understanding. People who are saying that there is no understanding in Chat GPT don't understand what's going on here.

11:38
Speaker 1
Other AI experts disagree with Yashua's conclusion that Chat GPT has the ability to understand. But Yashua is very concerned about what he sees happening with AI. That's next.

11:58
Speaker 3
We've all felt left out, and for people who move to this country, that feeling lasts more than a moment. We can change that. Learn how@belongingbeginswithus.org brought to you by the ad council.

12:19
Speaker 1
Yashua has seen AI come a long way, and recently he's been worried about how powerful the technology is getting. Some of his worries go back to the kind of Sci-Fi stories that fascinated him growing up.

12:33
Speaker 2
Some of the Sci-Fi stories are, amazingly, to the point of some of those concerns. So one that many people have seen is 2001 Space Odyssey with the HaL 9000 computer. Hello, Hal. Do you read me? You read me, Hal?

12:49
Speaker 1
In the movie 2001 A Space Odyssey, a team of astronauts comes to believe that their AI, named Hal, has been malfunctioning. So they decide to turn it off. But Hal anticipates their plan.

13:03
Speaker 2
Open the pod bay doors, Hal. I'm sorry, Dave. I'm afraid I can't do that.

13:10
Speaker 1
And Hal tries to kill them.

13:13
Speaker 2
Of course, it was never programmed to kill people. But as soon as you have a machine that has as a goal to preserve itself like we do, and if you think that somebody wants to turn you off, you're probably going to defend yourself.

13:28
Speaker 1
And you think AI could get there?

13:30
Speaker 2
I don't know, but I don't see any reasoning flaw in this story.

13:36
Speaker 1
It's plausible.

13:37
Speaker 2
It's plausible. Exactly. That's my concern.

13:41
Speaker 1
Yashua's had other concerns about AI for a while related to social media and disinformation and how AI can manipulate.

13:50
Speaker 2
AI has been used know in the last decade heavily for advertising, for being able to target just the right message to you, given the information that's available to the computer to make you change your mind. And of course, it's been used not just for advertising, but for recommendations, which could be useful in Facebook or e commerce. But it could also be a little bit disturbing. We don't want to be manipulated sort of behind our back. But now, if we have machines that can manipulate language as well as us, combine that with the commercial objectives of influencing people one way or another, for one reason or another, well, that's scary. So I was already concerned about AI in advertising, and then I realized, oh, there's a whole floodgate that's opening up that could be very dangerous.

14:44
Speaker 1
At the start of the year, Yashawa's concerns about AI grew more urgent.

14:50
Speaker 2
We have essentially reached that milestone that we have machines that master language. They don't necessarily have all the human abilities, but they have that one, which is crucial because it's the entry point into human culture, and that's what makes it so powerful. It can take advantage of everything that's written. We might be much closer than we anticipated to human level intelligence. And this is not what I was envisioning before chat.

15:20
Speaker 1
GPT OpenAI, the company that makes Chat GPT, didn't respond to our request for comment about Yashua's concerns. Earlier this year, Yashua joined with hundreds of other high profile AI researchers and tech leaders to write an open letter. They publicly called for a pause in AI tech development.

15:44
Speaker 2
I signed the letter at the end of March saying, oh, why don't we kind of slow down a little bit, make sure we understand what we're doing.

15:51
Speaker 1
Why did you feel like that was an important thing to sign?

15:55
Speaker 2
So I didn't really think that those companies would take a pause, but I thought it was important to speak up and to say that we should be cautious, that we need governments to step in. And I know that it takes time, and so the letter was not just about this pause thing, but it was alerting public opinion.

16:17
Speaker 1
The letter calls for more AI regulation and calls for AI research and development to focus on making systems more accurate, safe, and trustworthy. In May, Yahshua also signed another letter which claims that AI poses an extinction risk as great as pandemics and nuclear war.

16:37
Speaker 2
So the biggest catastrophe are these loss of control scenarios which really we don't understand well and we need to understand better. There's another kind of scenario which people don't talk too much about, and that is the excessive power concentration. So what do I mean? Maybe we find the ways to make AI safes, in other words, that we don't lose control of it and it does what we want, at least that the human controlling it gets what it wants first. They could be very rich, so they could acquire economic dominance. And once you're super rich, you can also acquire political power. And eventually there is an incompatibility between concentration of power and democracy. Democracy is about sharing power. If a few people decide everything, that's not democracy.

17:32
Speaker 1
For someone who spent his entire career working to advance AI technology, Yahshua says the shift in his own thinking has felt pretty destabilizing.

17:43
Speaker 2
It's difficult to make that shift because I had always seen my work as something positive for the world and starting to think, wait, if we get to human level intelligence and potentially surpass it, this could be harmful. Is hard to digest if you've seen yourself for all your life as working on something essentially good. I've always been saying we should be careful about social impact. But mostly I was very positive about technology and not like someone who's been spending their life talking about doom, but rather somebody who's been arguing that we should develop more of it in order to help us address many of the challenges that we have.

18:40
Speaker 1
Why is that fear true this time?

18:43
Speaker 2
Because we've reached an unexpected milestone. We are on a trajectory that's going quickly towards human level intelligence, and I don't think that we have the wisdom and the guardrails to handle this properly.

19:15
Speaker 1
Joshua, thank you.

19:17
Speaker 2
Pleasure.

19:33
Speaker 1
That's all for today. Tuesday, December 19. The journal is a coproduction of Spotify and the Wall Street Journal. If you want to learn more about AI, we have a series for you. It's called artificial the open AI Story. The first two episodes are already in your feed, and we're going to drop two more in January. Check it out. We'll link to it in our show notes. Thanks for listening. See you tomorrow.