Monday, June 29, 2009

Also, Consciousness vs. Memory

"Real time taps into consciousness, search taps into memory." Erick Schonfeld's recent TechCrunch post is illuminating. Beyond what Erick discussed of the real-time search, however, there is some more subtle distinction between the two that is worth of being thought. Hence here are my two cents.

Memory is the reserved and refined consciousness. Unlike consciousness that is often fuzzy and even pointless, memory generally has its definite theme and well-organized. We remember for a reason. The reason thus forms the backbone of memory. To search a particular piece of memory out of many, it is possible for us to figure out the algorithms disclosing the backbones. This is what the search engines have done and continue to improve.

Since real time message is indeed about the genuine consciousness in contrast to the refined consciousness (i.e., memory), the previous matrix of Web search no longer sustains. The genuine consciousness itself often lacks of reason. Most of the time it simply tells the undigested truth and the unconscious feeling. Therefore, it is impossible to "rank" the genuine consciousness the same way as if we have ranked (successfully?) the refined consciousness.

The thinking we have done points us to a new hint of effective real-time information search. In fact, we might not call it information search, but information refining. In contrast to search the real time messages, what the engine really needs to do is to refine the messages with respect to the pre-defined channels. By squeezing the water out, the refining process can eventually bring the high quality search results back to the information seekers.

The traditional/standard Web search delivers the direct access to the embodied, refined-already human consciousness. The real-time search then should direct the access to the procedures of refining the embodied, genuine human consciousness. This might be the intrinsic distinction between the two.


Kingsley Idehen said...


Nice and insightful as per usual.

You give search engines too much credit though :-) Search Engines have no clue re. dissambiguation of requests based on entity type and properties :-)

They don't comprehend individuality of the searcher.

I assume you've seen what we've been demonstrating at:


Using #1 or #2 above, with pattern: Camcorder, then use "Type" with the navigator to disambiguate the initial results; then use "show values with distinct count" or "show values" to see what you seek etc.. Of course, if you are scoped to what you want via "Type" filtering, add "Property" filtering to the mix etc..

Note: The UI isn't great, but that will be fixed soon.


Yihong Ding said...

thank you, Kingsley.

The work is a great initiative. The UI is not very straightforward at present. But as you said, the concept it reveals is great. We need to have better disambiguation of the requests in order to obtain answers in higher quality.

One interesting issue of semantic search is the mapping between intents in contrast to the mapping between requests and answers. That is, although the eventual goal is to figure out the mapping between the requests and the answers, we may need to first map the intent of the information producers and the intent of the information seekers so that the semantic search engine may achieve a better quality. I believe this point is generally overlooked so far. Without the filtering of the intent mapping, the direct quest-answer semantic mapping is likely too hard a problem to be resolved in any unrestricted domain.

By the way, the intent is the reason of memory I discussed in the post. I may have given too much credit to the present search engines. ;-) But ideally all feasibly successful search engines must have done more or less intent match in the level that matches the quality of its search results. In this sense, the evolution of search engines is probably more essentially the evolution of how we may successfully map the intent of the information producers and the information seekers.