Knowledge processing

It's always the same pattern: some well-known figure comes up with some idea and everybody jumps onto the bandwagon. I have the strong suspicion that media continues to fall into one of the famous logical fallacies, appeal to authority. Or perhaps it's just clever marketing. This time the hype is around a new project by the physics guy Stephen Wolfram of Mathematica fame: Wolfram alpha is coming and could be as important as Google. To cut a long boring blog entry short, I wouldn't hold my breath.

Question answering is a quite old sub discipline of computational linguistics, which nethertheless has seen a lot of progress in recent years. Still it happens to be a pretty hard task even in closed domains or for a given training set (see results of various TREC conferences, where TREC is an acronym for text retrieval conferences). Question answering in the open domain, as Wolfram alpha seems to address, is not one but multiple magnitudes harder: all of a sudden you no longer have a controlled terminology for queries and the amount of information you have to index and search is unbelievable.

There have been attempts in the past to deal with this problem. One particular well-known approach was or is CyC, which tried to build a huge knowledge base of every day knowledge. There have been several attempts to use Wordnet and more recently Wikipedia as a source of answers to questions. Even Microsoft tried to build a knowledge base from the data of its Encarta product. So, why don't we have already a well functioning open domain question answering system if people are trying to build one for like fifty years? See above: because it's really hard. Think about the parts involved: Information extraction in itself is not easy. Query parsing, as easy as it sounds, isn't trivially, either. Matching a query to an extracted piece of information usually requires a sophisticated system of knowlege representation or a similarly sophisticated statistical system. And text generation isn't a piece of cake either.

So what is it that Wolfram alpha makes different? From the very fuzzy amount of information we can see it's really hard to judge but I have very serious doubts that they've found the holy grail of QA. Unfortunately, we haven't seen any tests of their system but I guess there's a reason for it.
As for instance can be seen by the often ridiculous search results that the previous-google-killer-hype cuil offers, overcoming real world trouble like filtering out irrelevant or false data can be a major obstacle. Wolfram alpha of course doesn't have to filter out irrelevant web pages but it has a problem that is probably even greater: to filter out false claims on input data (assuming that they're operating on public available data) because otherwise they end up with wrong answers which would be disastrous. But on which grounds could you automatically filter out "wrong facts"? You would already have to know the "correct" ones.
So this leaves us with a lot of handcrafting, say to build a knowledge base of facts which they can answer. However, we've seen in the past that any handcrafted knowledge base requires vast resources and constantly so -- which is the major reason why wikipedia is a real problem for traditional encyclopedia publishers. Now remember that with Cyc there already has been an attempt to build up such a knowledge base and they've been working on it for roughly twentyfive years now and it's still not a system that is useful in reality (see, for instance the list of criticisms of the Cyc project on the wikipedia page on CyC.

But let's go back to that sensational article linked to above: in all seriousness, it's quite unlikely that even if they can build a system that can answer a lot of questions they're gonna get as important as Google is. Besides the fact that Google has enourmous resources including a lot of guys who know a lot about computational linguistics and is hence likely to come up with a similar system if necessary, Google has not been only a simple search engine for quite some time now. Google nowadays is in no way comparable to what it was ten years ago, their major service they provide is information access in a large variety of ways, including multiple media sources, integration of social interaction services . Fact retrieval and question answering (mainly based on texts) is certainly important but information access encompasses a lot more.

I'm not one of the people to get over-excited by the new US presidency, hence I normally wouldn't have any reason to say anything about it here. As a computational linguist, however, I can't let ReadWriteWeb's word cloud comparison of Obamas inaugeral speech to former speeches go unnoticed. From a casual look, it seems as if Bush communicated a lot clearer what his presidency would be about, at least looking back on the last years. Of course, all that talk about liberty and freedom was probably just advance justification for the aggressive actions to come. The word cloud analysis of Obama looks much broader but also much more unspecific to me -- which matches the image I got from the media pieces of his previous election speeches, too. It will be very interesting to see if Obama can fulfill all the wishful thinking people approach his presidency with (I wouldn't hold my breath, though) -- and, in some future time, how one might look back on that word cloud and which interpretation one is going to associate with all these terms.

Today I stumbled upon a list of top ten semantic web products of 2008 (according to ReadWriteWeb), which is interesting alone because of the varying range of the products listed. There is for instance a blogging helping browser extension (Zemanta) right next to an API providing semantic analysis of texts (OpenCalais). Another interesting point to me is that there seems to be somewhat contradictory directions: while some, e.g. Yahoo with it's SearchMonkey, seem to go into the direction of open access, others like Hakia are strictly closed-shop approaches. It will be very interesting to see where future development will be headed, given that knowledge is not necessarily a unique thing -- as can be easily seen in the many different ontologies that have been built or in the edit wars going on in Wikipedia. Probably these differences might go hand in hand with company size or profit -- the odd start up is as well on the list as large corporations. I don't like to sound like Cassandra, but given the recent financial crisis, I sincerly hope that the precious and small commercial flowers won't starve in another AI winter 2.0.

But perhaps there is a chance that open source approaches bring some movement -- today I also happened to learn about Nepomuk, a "Networked Environment for Personalized, Ontology-based Management of Unified Knowledge" -- that happens to be integrated in the latest KDE version 4.0. Although, from what I hear from my KDE using friends, none of them seems to love that particular integration, but I can't judge whether that has more to do with the backend or the integration.

James Hague blogs about how writing a spellchecker used to be a major feat of software engineering, the main point being that memory was a main problem that's no longer an issue today. While that is certainly true, the two issues (memory, spellchecking problem) are not as much related as James claims.

Without making it explicit, he makes the assumption that for spell checking to work you have to use a dictionary consisting of entire (correctly spelled) words. That would have been a very unwise way to go in the 80ties and it still is nowadays. /usr/share/dict contains roughly 240.000 words and you find that impressive? I don't. The real number of words of a language is much larger, which is easier to see in languages like German where you can easily build compound words. Hence, linguists typically try to capture as much information as they can in word building rules.

This will leave you with three things: a dictionary of words which are not built according to the generally applicable rules (for instance necessary for foreign words prominent in your language of choice), and a set of stems, basic building blocks of words. Quite to the contrary of James' claim that 3-5 lines of Perl are everything you need, the thing to keep in mind here is that this is the only way to solve the "spellchecking problem": any full-form approach to spellchecking is doomed to fail because a language's vocabulary does not consist of a closed set of words. How else could publishers of dictionaries go on selling new versions of their dictionaries every year? Another point is that in some languages (e.g. German) some aspects of spelling might be dependent on context (grammatical and semantical context). I.e. in order to be able to decide whether it's correct to write "Fahren" you need to have a basic understanding of the grammatical composition of the sentence it appears in: it might be a noun derived from "fahren" (to drive) or it might be situated at the beginning of a sentence (and sentence boundary detection is not always trivial either). Now, if your spellchecker just checks a dumb list of words it can't decide whether "Fahren" is correct or wrong.

With regard to the implementation issue, using a non-full form approach also helps: The nice thing is that you don't have as much redundancy and you can spellcheck a lot of words you've never heard of just by following the rules. Unfortunately, the number of stems is still huge if you want to do achieve a good rate, so of course clever tricks to deal with the list of stems is still required. There are of course still different routes you might take to implement spellchecking: what ispell or aspell are doing, for example, is mostly using a full-form approach using only minor modifications to recognize some simple word production rules. This is not what you will find in state-of-the-art spell checkers.

This started out as a reddit reply to Alex Iskold's article about semantic search engine technology. Alex suggests that the reason why we haven't seen the semantic web as overtaking as the search technology of today is because of two points: that they won't provide better search results today and that the current semantic search engines have UI problems. At the same time, he claims that (wrt. to the search results) "emantic search is going to be big and it is going to help us answer questions that we simply cannot answer today - complex, inferencing queries asked over the entire web as if it was a database." I think that the UI point is irrelevant, or perhaps only relevant at this point in time.

I agree with the search result point, but I'm much more sceptical about the future results to expect.