We can’t trust AI systems built on deep learning alone

11/8/2019

We can’t trust AI systems built on deep learning alone

We can’t trust AI systems built on deep learning alone usm systems Sep 27 · 8 min read

Gary Marcus is not impressed by the hype around deep learning. While the NYU professor believes that the technique has played an important role in advancing AI, he also thinks the field’s current overemphasis on it may well lead to its demise.

Marcus, a neuroscientist by training who has spent his career at the forefront of AI research, cites both technical and ethical concerns. From a technical perspective, deep learning may be good at mimicking the perceptual tasks of the human brain, like image or speech recognition. But it falls short on other tasks, like understanding conversations or causal relationships. To create more capable and broadly intelligent machines, often

https://medium.com/@swetha23/we-cant-trust-ai-systems-built-on-deep-learning-alone-383018353331

1/6

11/8/2019

We can’t trust AI systems built on deep learning alone

referred to colloquially as artificial general intelligence, deep learning must be combined with other methods.

When an AI system doesn’t truly understand its tasks or the world around it, that could also lead to dangerous consequences. Even the smallest unexpected changes in a system’s environment could make it go awry. Already there have been innumerable examples of this: hate speech detectors that are easy to fool, job application systems that perpetuate discrimination, and self-driving cars that have crashed, sometimes killing the driver or a pedestrian. The quest for artificial general intelligence is more than an interesting research problem. It has very real-world implications.

In their new book Rebooting AI, Marcus and his colleague Ernest Davis advocate for a new path forward. They believe we are nowhere close to achieving such general intelligence, but they are also confident that we can get there eventually.

I spoke with Marcus about the weaknesses of deep learning, the lessons the field can borrow from the human mind, and why he’s optimistic.

It has, and it will generate even more. But there are lots of problems that narrow AI just doesn’t seem very capable of. Things like conversational natural-language understanding and general assistance in the virtual world, or things like Rosie the robot that might be able to help you tidy your home or cook you dinner. Those are just way outside of the scope of what we can do with narrow AI. It’s also an interesting empirical question about whether narrow AI can get us to safe driverless cars. The reality so far is that narrow AI has a lot of problems with outlier cases, even for driving, which is a fairly constrained problem.

More generally, I think we would all like to see AI help us make new discoveries in medicine at scale. It’s not clear that current techniques are going to get us to where we need to be, because biology is complicated. You really need to be able to read the literature. Scientists have causal understandings about how networks and molecules interact; they can develop theories about orbits and planets or whatever. With narrow AI, we can’t get machines to do that level of innovation. With general AI, we might well be able to revolutionize science, technology, medicine. So I think working toward general AI is very much a worthy project.

https://medium.com/@swetha23/we-cant-trust-ai-systems-built-on-deep-learning-alone-383018353331

2/6

11/8/2019

We can’t trust AI systems built on deep learning alone

General AI is about having AI be able to think on the fly and resolve new problems on its own. This is as opposed to, let’s say, Go, where the problem hasn’t changed in 2,000 years.

General AI also ought to be able to work just as comfortably reasoning about politics as reasoning about medicine. It’s the analog of what people have; any reasonably bright person can do many, many different things. You take an undergraduate intern and, within a few days, have them work on essentially anything from a legal problem to a medical problem. It’s because they have a general understanding of the world, and they can read, so they’re able to contribute to a very wide range of things.

The relation between that and robust intelligence is if you’re not robust, you’re probably not really going to be able to do the general thing. So in order to build something that’s reliable enough to deal with a world that’s constantly changing, you probably need to at least approach general intelligence.

But you know, we’re pretty far from that right now. AlphaGo can play very well on a 19x19 board but actually has to be retrained to play on a rectangular board. Or you take your average deep-learning system, and it can recognize an elephant as long as the elephant is well lit and you can see the texture of the elephant. But if you put the elephant in silhouette, it might well not be able to recognize it anymore.

In cognitive science, we talk about having cognitive models of things. So I’m sitting in a hotel room, and I understand that there’s a closet, there’s a bed, there’s the television that’s mounted in an unusual way. I know that there are all these things here, and I don’t just identify them. I also understand how they relate to one another. I have these ideas about how the outside world works. They’re not perfect. They’re fallible, but they’re pretty good. And I make a lot of inferences around them to guide my everyday actions.

The opposite extreme is something like the Atari game system that DeepMind made, where it memorized what it needed to do as it saw pixels in particular places on the screen. If you get enough data, it can look like you’ve got understanding, but it’s actually a very shallow understanding. The proof is if you shift things by three pixels, it plays much more poorly. It breaks with the change. That’s the opposite of deep understanding.

https://medium.com/@swetha23/we-cant-trust-ai-systems-built-on-deep-learning-alone-383018353331

3/6

11/8/2019

We can’t trust AI systems built on deep learning alone

I’m just sort of flabbergasted at how little appreciation the deep-learning community, in general, has for that.

There are a few. First, classical AI actually is a framework for building cognitive models of the world that you can then make inferences over. The second thing is, classical AI is perfectly comfortable with rules. It’s strange sociology right now in deep learning where people want to avoid rules. They want to do everything with neural networks and do nothing with anything that looks like classical programming. But there are problems that are routinely solved this way that nobody pays attention to, like making your route on Google maps.

We actually need both approaches. The machine-learning stuff is pretty good at learning from data, but it’s very poor at representing the kind of abstraction that computer programs represent. Classical AI is pretty good at abstraction, but it all has to be handcoded, and there is too much knowledge in the world to manually input everything. So it seems evident that what we want is some kind of synthesis that blends these approaches.

I think there’s another way of making the point, which is: every cognitive system that we have is really doing a different thing. Similarly, the counterparts in AI need to be designed to tackle different problems that have different characteristics.

Right now people are trying to use kind of one-size-fits-all technologies to tackle things that are really fundamentally different. Understanding a sentence is fundamentally different from recognizing an object. But people are trying to use deep learning to do both. These are qualitatively different problems from a cognitive perspective, and I’m just sort of flabbergasted at how little appreciation the deep-learning community, in general, has for that. Why expect that one silver bullet is going to work for all of that? It’s not realistic, and it doesn’t reveal a sophisticated understanding of what the challenge of AI even is.

It’s again a place where deep learning is not particularly well suited. Deep learning doesn’t give explanations for why things happen but, rather, a probability for what might happen in a given circumstance.

Nature builds the first draft, the rough draft. Then learning revises that draft throughout the rest of your life. https://medium.com/@swetha23/we-cant-trust-ai-systems-built-on-deep-learning-alone-383018353331

4/6

11/8/2019

We can’t trust AI systems built on deep learning alone

The kind of stuff that we’re talking about-you look at some scenarios, and you have some understanding about why it happens and what might happen if certain things were changed. I can look at the easel that the hotel television is on and guess that if I cut away one of the legs, the easel will tip over and the television is going to fall down with it.

That’s causal reasoning.

Classical AI gives us some tools for this. It can represent, for example, what a supportive relationship is and what’s falling over. I don’t want to oversell it, though. One problem is classic AI mostly depends on very complete information about what’s going on, whereas I just made that inference without actually being able to see the entire easel. So I’m somehow able to make shortcuts, inferring pieces of the easel that I can’t even see. We don’t really have tools that can do that yet.

For humans, by the time you’re born, your brain is actually very elaborately structured.

It’s not fixed, but nature builds the first draft, the rough draft. Then learning revises that draft throughout the rest of your life.

A rough draft of the brain already has certain capabilities. A baby ibex just a few hours old can scramble down the plane of a mountain without making mistakes. Clearly it has some understanding of three-dimensional space, its own body, and the interrelation between the two. Pretty sophisticated stuff.

This is part of why I think we need hybrids. It’s hard to see how we could build a robot that functions well in the world without analogous knowledge there from the start, as opposed to starting with a blank slate and learning through enormous, massive experience.

For humans, our innate knowledge comes from our genomes that have evolved over time. For AI systems, they have to come a different way. Some of that can come from rules about how we build our algorithms. Some of it can come from rules about how we build the data structures that those algorithms manipulate. And then some of it might come from the knowledge that we just directly teach the machines.

Because I think it’s the whole ballgame right now. I think that we’re living in a weird moment in history where we are giving a lot of trust to software that doesn’t deserve that

https://medium.com/@swetha23/we-cant-trust-ai-systems-built-on-deep-learning-alone-383018353331

5/6

11/8/2019

We can’t trust AI systems built on deep learning alone

trust. I think that the worries that we have now are not permanent. A hundred years from now, AI will warrant our trust-and maybe sooner.

But right now AI is dangerous, and not in the way that Elon Musk is worried about. But in the way of job interview systems that discriminate against women no matter what the programmers do because the techniques that they use are too unsophisticated.

I want us to have better AI. I don’t want us to have an AI winter where people realize this stuff doesn’t work and is dangerous, and they don’t do anything about it.

Right, the book is very short-term pessimistic and very long-term optimistic. We think that every problem that we described in the book can be solved if the field takes a broader view about what the right answers are. And we think that if that happens, the world will be a better place.

Want to know more about AI services then have a free consultation for USM systems

Arti cial Intelligence

Deep Learning

Data Science

Machine Learning

Ai Services

About

https://medium.com/@swetha23/we-cant-trust-ai-systems-built-on-deep-learning-alone-383018353331

Help

Legal

6/6