measuring a computer’s iq the singularity way

June 4, 2010

Contrary to what you might think from my posts about the notion of the Technological Singularity, I do take the claims made by Singularitarians quite seriously and take time to look at their arguments. Now, often times I’m reading papers dealing with very abstract ideas, few tangible plans for a particular system, and rather vague, philosophical definitions which have little to do with any technical designs. Recently, however, I took a look at the work of Shane Legg at the recommendation of Singularity Institute’s Michael Anissimov, and found some real math into which to sink my teeth. Legg’s goal was to come up with a way to measure intelligence in a very pure and abstract form, especially as it applies to machines, and provide a better definition of Singularitarian terms like “super-intelligence,” creating a very interesting paper along the way. But, as you probably guessed already, there are some issues with his definitions of intelligence and what his formula truly measures…

To make a long story short, Legg is measuring outcomes of an intelligent agent’s performance in a probability game based on what strategies should yield the best results and the biggest rewards over time. And that’s an effective way to tackle intelligence objectively since in the natural world, bigger and more complex brains with new abilities are encouraged through rewards such as food, water, mating, luxuries, and of course, a longer, better lifespan. But there are a few problems in applying a formula to measure performance encouraged by a reward of some sort to a bona fide intellect, especially when it comes to AI. Humans program the strategy the machine will need to meet these intelligence tests and are basically doing all the work. Even if the machine in question does have to learn and adapt, it’s following human algorithms to do so. Compare that to training any intelligent animal which learns the task, figures out exactly what it needs to do and how, then finds shortcuts that either maximize the reward or reduce the time between rewards. Legg’s formula can measure outcomes in both cases, but what it can’t measure is that a computer has been “pre-wired” to do something while mice, dogs, or pigs, for example, effectively “re-wired” their brains to accomplish a new task.

The paper is keenly aware that people like me would question the “how” of the measured outcomes, not just the grading curve and circumvents this problem by saying that the formula in question is concerned only with the outcomes. Well that hardly seems fair, does it? After all, we can’t just ignore the role of creativity or any other facets of what we commonly call intelligence, or make the task of defining and building AI easier with various shortcuts meant to lower the bar for a computer system we want to call intelligent. Just as Legg’s preamble points out, using standardized IQ tests which deal with certain logical and mathematical skills isn’t necessarily an accurate summation of intelligence, just some facets of it that can be consistently measured. To point this out, then go on to create a similar test taking it up one notch in abstraction and say that how well a subject met certain benchmarks is all that matters, doesn’t seem to break any new ground and countering a pretty important question by saying that it’s just out of the work’s scope seems like taking a big shortcut. Even when we cut out emotions, creativity and consciousness, we’re still left with a profound difference between an intelligent biological entity and a computer. Although patterns of neurons in brains share striking similarities with computer chips, biology and technology function in very different ways.

When we build a computer, we design it to do a certain range of things and give it instructions which predict a range of possible problems and events that come up during an application’s execution. If we can take Legg’s formula and design a program to do really well at the games he outlines, adopting the strategies he defines as indicative of intelligence, who’s actually intelligent in this situation? Legg and programmers who wrote this kind of stuff for a typical homework assignment in college, or the computer that’s being guided and told how to navigate through the IQ test? Searle’s Chinese Room analogy actually comes into play in this situation. Now if we were compare that to humans, who are born primed for learning and with the foundations of an intellect, playing the same games, the fundamental process behind the scenes becomes very different. Instead of just consulting a guide telling them how to solve the problems, they’ll be actively changing their neural wiring after experimenting and finding the best possible strategy on their own. While we can pretend that the how doesn’t matter when trying to define intelligence, the reality is that living things like us are actually guiding computers, telling them how we solve problems in code, then measuring how well we wrote the programs. To sum it up, we’re indirectly grading our own intelligence by applying Legg’s formula to machines.

The same can be said about a hypothetical super-intelligence which we’ve encountered before in a paper by futurist Nick Bostrom where it was very vaguely and oddly defined. Legg’s definition is much more elegant, requiring that in any situation where an agent can earn a reward, it finds the correct strategy to get the most it possibly can out of the exercise. But again, apply this definition to machines and you’ll find that if we know the rules of the game our AI will have to beat, we can program it to perform almost perfectly. In fact, when talking about “super-human AI,” many Singularitarians seem to miss the fact that there are quite a few tasks in which computers are far better than humans will ever be. Even our ordinary bargain bin netbook can put virtually any math whiz to shame. Try multiplying 1.234758 × 10^33 by 4.56793 × 10^12. Takes a while, doesn’t it? Not for your computer which can do it in a fraction of a millisecond though. Likewise, your computer can search more than a library’s worth of information in a minute while you may spend the better part of a few months to do the same thing. Computers can do a number of tasks with super-human speed and precision. That’s why we use them and rely on them. They reached super-human capabilities decades ago but because we have to write a program to tell them how to do something, they’re still not intelligent while we are.

In fact, I think that by using computers and outsourcing detail-oriented, precision and labor intensive tasks for which evolution didn’t equip our brains is in itself a demonstration of intelligence in both logical and creative realms. In our attempts to define computer intelligence, we need to remember that computers are tools and if we didn’t have access to them, we could still find ways of going about our day to day tasks while any computer without any explicit directions from us would be pretty much useless. Now, when computers start writing their own code without leaving a tangled mess and optimizing their own performance without any human say in the matter, then we might be on to something. But until that moment, any attempt to grade a machine’s intellect is really a roundabout evaluation of the programmers who wrote its code and the quality of their work.

[ illustration by Hossein Afzali ]

Share
  • http://www.vetta.org Shane Legg

    “Humans program the strategy the machine will need to meet these intelligence tests and are basically doing all the work.”

    Evolution shaped and “designed” the structure of your brain. One of the algorithms it put into your brain is temporal difference learning. It’s a kind of reinforcement learning algorithm that involves a number of brain areas, in particular parts of your dopaminergic system. If I understand you correctly: a machine using temporal difference learning cannot be intelligent because the algorithm was put there by a human. While for your brain using temporal difference learning is ok because it was put there by evolution?

    “a computer has been “pre-wired” to do something”

    You can pre-wire a system to solve a problem when the problem belongs to a limited class. However, the universal intelligence measure spans the set of all possible problems. You can’t pre-wire for that. It would require you coming up with infinitely many solutions to all possible problems, in all possible worlds, in all possible situations. The concept of pre-wiring breaks down in this situation.

    “we can’t just ignore the role of creativity”

    Does creativity improve your ability to deal with some kinds of problems? If the answer is yes, then because the universal intelligence measure spans all possible problems it is already measuring the agent’s creativity.

  • Greg Fish

    … While for your brain using temporal difference learning is ok because it was put there by evolution?

    Well, no, not quite. Sure, we could say that a human programming an algorithm into computers could be viewed in the same way as evolution’s shaping of a brain. But a brain is plastic and actively comes up with new ways to get something, while robots and computers only do what their programs tell them. Moreover, to learn, I don’t need to give a program encouragement. I simply tell it to cycle through its instructions until I’m satisfied with the results.

    If anything, I thought your universal intelligence formula would be great at defining an intelligent behavior in a totally alien environment and shows that intelligence itself is not something with which we’re zapped, but an abstract measurement that comes in a wide range of degrees. However, I don’t think it applies to machinery.

    You can pre-wire a system to solve a problem when the problem belongs to a limited class. However, the universal intelligence measure spans… all possible problems.

    Yes, and I certainly can’t pre-wire for that or write enough programs to cope with any possible problem to be thrown at the machine. However, knowing how you measure the level of the computer’s intelligence, one could program a very generic approach for most strategy games and you actually detail these basic strategies in your paper. The problem then becomes how to actually communicate something indeterminate to a computer, but that’s more of a technical challenge.

    because the universal intelligence measure spans all possible problems it’s already measuring the agent’s creativity.

    In my interpretation of the math, you’re evaluating the rate of an agent’s success in a problem in which it has to find a way to get a reward. Now, if there was something to measure how many novel, successful solutions are being employed by the agent in the calculation, then I’d agree with this statement. So creativity is playing a role, but it isn’t being actively measured.

  • http://www.vetta.org Shane Legg

    Your criticisms that computers can only do what we program them to do is a criticism of the possibility of machine intelligence in general, rather than anything specific to the universal intelligence measure which does not assume that the agent is computable.

    That aside, you say that computers only do what we program them to do, and humans do not have this limitation. Thus, is it fair to say that you believe that humans and other animals can do things that computers cannot do, no matter how they are programmed?

    “one could program a very generic approach for most strategy games”

    That would be a start, but there are many things that a simple strategy won’t be able to handle. For example, does it allow the agent to write an award winning romance novel? A simple strategy on a real computer with limited resources will only deal with a limited subset of environments and thus will have a limited universal intelligence.

    “So creativity is playing a role, but it isn’t being actively measured.”

    Yes, I only care about creativity to the extent that it helps an agent achieve in some situation. I do not care about any aspects of creativity (however you define it) that are of no use ever in any situation.

  • Greg Fish

    Your criticism that computers can only do what we program them to do is a criticism of the possibility of machine intelligence in general…

    Absolutely. All my applications of it to computers are done within a narrow context of trying to objectively define a potential AI system.

    … is it fair to say that you believe that humans and other animals can do things that computers cannot do, no matter how they are programmed?

    Technically, we could program a robot to simulate how an animal behaves and that’s already been done numerous times. But I think we need to draw a clear line between mimicry on command and self-motivated action. True, it’s outside your paper’s scope but in the context in which Michael referred to your work, I thought it mattered.

    A simple strategy on a computer with limited resources will only deal with a limited subset of environments and thus will have a limited universal intelligence.

    Agreed. We could only boost computing power so much until physics starts holding us back, and even then, they’ll only be able to handle a limited number of strategies for solving real world problems due to their inner workings.

  • http://www.vetta.org Shane Legg

    I’m confused about your position.

    Do you believe that is it possible, at least in theory, to program a computer so that its *behaviour* is identical to a human?

    For example, it could go to college, be taught particle physics, argue with you on your blog, practice pick up lines in a bar, learn to use new technologies that didn’t exist when we programmed it, get married and go on to have an outstanding career as a stand up comic and win a Nobel prize for a new discovery it made?

  • Greg Fish

    Do you believe that is it possible, at least in theory, to program a computer so that its *behaviour* is identical to a human?

    Do I believe that it’s technically plausible to program a machine to mimic humans if we had a powerful enough CPU and sufficient resources to write the millions of lines of code that would require? I think it would be highly impractical and far too expensive for even the best funded computer science departments in the world to do, but I can’t honestly discount that it’s theoretically plausible.

    You can program a computer to understand language, which is no easy feat, but it’s an emerging necessity for search engines and projects like WolframAlpha to make sense of user requests. Once you can do that and impart strategies for learning, like an ANN or content recognition systems you can take your system pretty far. The end result would probably look like a huge server farm rather than anything even remotely like an android, but it could mimic human behavior to a very significant degree.

    As for things like jokes, we already have computers that can understand sarcasm at a success rate comparable to humans and a program that creates puns. However, I struggle to see how we could impart a computer with enough wit to be a comic since it’s not a talent many people even have or scientific creativity that would let it come up with a revolutionary scientific advancement. Then again, we don’t even know how we do it ourselves…

    But the big question is whether all the guiding and prodding that will be required for all this to happen will result in an actual intelligence. I would submit that mimicry, no matter how elaborate, is still mimicry, and our theoretical AI system would be lost if we were to switch up the language it’s build to understand. New words, new rules of grammar, new ideas, new alphabets all mean that the core of the system will have to be re-written from scratch. Humans could figure out nonverbal cues which make up the vast majority of all communication and are intuitive to us, but indecipherable to a computer which only understands on/off signals and logic gates.

  • http://www.vetta.org Shane Legg

    I still can’t figure out if your answer to my question is a yes, or a no. Either it is possible *in theory* (we can worry about in practice later) to program a computer to be behaviourally indistinguishable from a human, or it isn’t.

    Which side do you take?

  • Greg Fish

    I was trying to be really cautious and detailed about my answer, so pardon me if it came off as a little obtuse.

    In one sentence, I would say that yes, we could theoretically program a computer to mimic human behavior so closely that it will pass the Turing test. Actually, I spent a good deal of my reply trying to explain how it might be done.

    So Dr. Legg, what’s your take on machine intelligence? Do you think it matters how an intelligent agent tackles problems or is the very fact that it can tackle them qualify the agent as intelligent?

  • http://www.vetta.org Shane Legg

    Ok, thanks, that helps.

    I think I understand where the problem lies here. In their text book Russell and Norvig point out that perspectives on what AI is trying to achieve fall into four main groups:

    1) Thinking humanly
    2) Acting humanly
    3) Thinking rationally
    4) Acting rationally

    4 seems to be the most common group among AI researchers, and it’s the group I belong to. From this perspective, intelligence *is* the ability to act rationally so as to achieve goals in the world (roughly stated). All that matters is the agent’s external behaviour. How the agent works inside has nothing to do with what intelligence is. Whether it is internally pre-wired, or mimicking, or what ever makes no difference. It could be powered by voodoo mice singing Christmas carols for all we care.

    You seem to belong to perspective 1 (or maybe 3). For example, if I came to you with an agent that I had programmed that was just like a human from the outside, you might not consider it to be intelligent. Even if it could interact in conversation with you in a normal way, you could send it to classes and it could learn to speak Chinese, etc… that’s still not enough to establish that it’s intelligent because behaviour alone is not enough. Aspects of how it works internally matter. Searle takes this position in his Chinese room argument, and Ned Block has a related perspective.

    Which is the “right” perspective? Well, this question doesn’t really make sense. What you want to achieve and what I want to achieve are simply different things. The confusing bit is that we both use the word “intelligence” to describe what we are talking about.

    Getting back to the original topic, the universal intelligence measure is designed to measure progress towards number 4. If that’s not your perspective, then it’s not a useful measure for you.

  • Greg Fish

    Actually, I would say that I’m interested in AI from points 1 and 3 because I see it as a way to delve further into researching the origins of cognition and see how a complex network of repeating circuits that we find in brains of all types and sizes can produce what we identify as sapient thought. What I’d build may not apply to biological entities, but I’m hoping it could trigger some new hypotheses.

    Thank you for your replies Dr. Legg. I had a lot of fun and a lot of things to think about.

  • http://weirdsciences.net bruceleeeowe

    A question for you Dr. Legg. How many times have you calculated the angle before taking a turn. I’m against silicon intelligence based on current aspects of AI. Providing sense and emotions to iron machines seems me highly improbable. Can’t you tell what algorithm will you use to produce such type of cognitive intelligence and emotions?

  • http://www.vetta.org Shane Legg

    You can call me Shane, everybody else does.

    You say you have a question, but it’s not clear to me what the question is. So let me take your first sentence as your question:

    “How many times have you calculated the angle before taking a turn.”

    It depends what you mean by “you”. Do you mean some part of my brain/body which I might or might not be consciously aware of? We can look in parts of primate motor cortex and see angle and direction information being computed before movement actions are taken. So I guess the answer to your question is, “all the time”.