The Implicit Web

(This article is cross-posted at ZDNet's Web 2.0 Explorer.)
(watch the article also in Chinese, translated by the author)

Implicit web is a new concept coined in 2007. Due to the first Defrag conference right now, discussion of this new term is timely.

Generally this concept implicit web intends to alert us a fact that besides all the explicit data, services, and links, the Web engages with much more implicit information such as which data users have browsed, which services users have invoked, and which links users have clicked. This type of information is often too boring and tedious to be human readable. So, inevitably, this type of information is only implicitly stored (if stored) on the Web. The implicit web intends to describe a network of this implicit information.

Implicitness Everywhere

Implicit information is everywhere. Implicit information on the Web is about things to which human web users have paid attention. For example, it is about which web pages are frequently read, how often they are read, and who read them. It is also about which services are frequently invoked, how often they are invoked, and who invoked them. Consider the number of web users and how many activities everybody has done daily on the Web, the amount of implicit information must be astonishing. The implicit information co-exists with every web page, every web service, and every web link. In short, great amount of implicitness co-exists with every little piece of explicitness on the Web.

Implicit does not mean insignificant or unimportant. By contrast, implicit web information is often valuable and even crucial in various situations. For example, implicit information of click rates can help editors decide which news are the most popular ones and thus they should put these news on the front page. In similar, the same type of implicit click rates can help salespeople decide which merchandises are among the greatest demanding and so they can arrange the next supply line.

Many companies have already started to collect implicit information and they take benefits from it. Alex Iskold had written a compact introduction on how some companies have utilized implicit information in their products. One well-known example is Amazon.com, which always lists related buyer recommendations with each of its online merchandise. "Customers Who Bought This Item Also Bought," many readers must be familiar to this label. And more importantly, many web users do care of the content underneath this label. This is a typical example of how implicit web information helps.

Amazon is not the only company that benefits from implicit information. Amazon is not one of the few companies that benefit from implicit information. In fact, nowadays almost every website that sells something, from baby toys to cars, has some back-end mechanism on analyzing the traffic (a typical implicit information) and adjust their sales plan based on the analysis. Implicitness is indeed everywhere.

Connect Implicitness

Implicitness is everywhere, but is fragmented everywhere. Implicit information on the Web is not connected. This is a problem.

Until now, implicit web information is generally separately stored, typically by individual companies. For example, both Gap.com and jcrew.com have their own stored visitor history but not shared to the other, although we may imagine that this information must be well connectible since both companies sell apparel and accessories. Someone may argue that Gap and J. Crew are competitors. So let us switch the pair to be Banana Republic and Victoria's Secret. The products of these two companies are well complement (in contrast to compete) to each other. But still the implicit information is isolated to itself, despite that both sides can benefit by connecting this independent implicit information. Readers can find many more this type of examples.

If sharing implicit information among big companies is still questionable (because these big boys hardly believe that they could get help from their little sisters), this type of sharing is much more critical to small websites. There are numerous individual sites that cannot utilize themselves well enough from their own implicit information because they are too small in size. At the same time, however, there are no effective way for them to share and find helpful implicit information, though everybody knows that there is plenty of this information on the Web.

All these discussions lead to one demand: we need the implicit web, which is not there yet. The goal of the implicit web is to defragment all the fragments of implicitness (where the name Defrag is gotten for the conference). But how can we indeed connect all the different types of implicitness on the Web to be a coherent implicit web? This is a grand challenge to the newly formed community of implicit web research. We do not have a clear answer yet.

No matter whatever, however, the solution to the question must be beyond web links. The implicit web engages with complex types of semantics. The amount of information on the implicit web is gigantic. The implicit Web is also very much dynamic. The traditional model of web link is too simple, too shallow, and too static to deal with all these challenges at the same time. We need big, creative thoughts to store and link all the implicitness.

The greatest potential problem to the implicit web is privacy. To companies, some implicit information may be too confidential to be shared. To individual persons, some implicit information may be too private to be public. We need innovative methods of privacy control on the implicit web.

Implicit Web in nutshell

In summary, I briefly list my beliefs about the implicit web.

1. The implicit web is a network that defragments every piece of implicitness on the explicit web, which is the generally known World Wide Web itself.

2. If the explicit web reveals the static side of human knowledge through posted data, services, and links, the implicit web reveals the dynamic side of human knowledge by recording how users access these data, services, and links.

3. The explicit web engages collective human intelligence. The implicit web engages collective human behaviors.

4. The implicit web is not part of the Semantic Web, but they are closely related. If the Semantic Web constructs a conceptual model of World Wide Web, the implicit web constructs a behavior model of World Wide Web.