A Case for Integrated AI

Scott Fahlman,   March 30, 2008
Categories:  AI    

The best presentation I have seen in the past few years about where AI is now and where we should be heading was Ron Brachman’s 2006 Presidential Address to the American Association for Artificial Intelligence (AAAI)[1] conference. A transcript of that talk was reprinted in AI Magazine (Winter 2006) and is available online. Everyone interested in AI should read this.

Given the occasion, the article necessarily contains some thoughts about the proper role of AAAI as an organization. Those may or may not be of interest to readers of this blog. But I believe that Ron’s arguments for the importance of an integrated attack on intelligence are right on the mark. As long as most AI researchers focus on only one small component of intelligence or one kind of problem, we will never have systems that exhibit human-like versatility in a complex, ever-changing environment.

The one area where I disagree with Ron is in his enthusiasm for evaluation and metrics in our work – a new-found enthusiasm, according to his article. It is true that our sponsors will often require detailed, quantitative performance measurements as a way to evaluate the success of the projects they fund. So we have to live with that.

It is also true that there are times when, if no scoring system were imposed on a research effort, we would have to invent one. That is often a good way to keep the project members focused and motivated, and there are times when a bit of competition (keeping score) is a good thing. In areas like statistical learning, appropriate metrics may be the only good way to tell if the system is working properly and doing something interesting.

However, I have seen too many promising research projects ruined or badly damaged by rigid metrics, rigidly applied. A “wild frontier” field like AI does not progress in slow and steady increments. Implementing a promising new idea usually requires that we take a step or two backward in our measurable performance before we get the great leap forward. Sometimes these new ideas will fail — that’s an important part of research. If we are in a situation where a particular performance metric must improve by a particular amount every year , with the threat of immediate project cancellation as the “incentive”, that creates a tremendous disincentive to try anything risky and innovative. I can think of no better way to ensure that a project will produce some routine progress, but no breakthroughs. And if the metrics are used to drive a competition – only the top few groups in each year’s competition will be funded for the next year – that guarantees that the competing groups will be secretive and unable to build on one another’s ideas.

Anyway, this blog is supposed to be about AI ideas and not about the management of research projects, so I’ll leave it at that. I do share Ron’s enthusiasm for “Grand Challenge” problems, whenever we can come up with good one. I’ll say more about that in a later post.

[1] The organization has since renamed itself the “Association for the Advancement of Artificial Intelligence”, keeping the AAAI acronym and properly reflecting AAAI’s evolution into an international organization.