Build new centrifugal momentum on the Web

Nicolas Carr shared a keen observation of a present trend on the Web---the centripetal force is currently driving the progress of the Web. An invisible force is pulling Web users away from the comparatively smaller Web sites and towards the bigger nodes sitting on the center of the Web. For instance, more Technorati blog search users now adopt Google blog search, and more users of Bloglines now switch to Google Reader. In short, the effect of Long Tail is decaying.

Nick's observation is illuminating. Innovation is the centrifugal force on the Web. Due to innovation, new Web sites can attract more users away from the large existing sites on the center of the Web. The more innovation the Web has, the stronger the centrifugal force is.

On the contrary, when the centripetal force starts to dominate, Web innovation is at its downturn. In particular, this time the downturn tells that the principles of Web 2.0 may no longer keep on generating the centrifugal momentum great enough to sustain the high-speed expansion of the Web. The Web evolution is entering a comparative contraction process (i.e., still expanding but in the low-speed mode).

Be note that this is the second time the Web enters a comparative contraction process. The last time was during the period of the dotcom bubble in which many small Web sites went out of business and the majority of Web users flowed back to the few major sites on the Web. After the dotcom bubble, however, the Web expansion re-accelerated in a new stage---Web 2.0.

Based on all the analysis, we should be confident to expect another new round of rapid Web expansion ahead despite of the present contraction. Now it is the time for us to build new centrifugal momentum on the Web.

The Age of Google (2): Open Sesame

(the previous installment: The Age of Google (1))

It is an old Arabic story. By accident, a poor Arabic young man named Ali Baba overheard a message spoken by a group of thieves---forty in total. From the message, he learned a tremendous amount of treasure hidden in a cave. The magic spell to open the cave was "Open Sesame." Ali Baba thus entered the cave with the secret spell and took some of the treasure home.

World Wide Web is the cave full of treasure. The search engine sites are the magic doors. The user-specified keywords are, however, the "open sesame."

Among all the magic doors, Google is the most marvelous one. Google has standardized the way of ranking online information. Before Google, we had diverse standards to rank the relevancy of information on the Web. Because of Google, the standards converged. This convergence is an important sign about the age of Google.

Subjective ranking vs. Objective ranking

In primary should ranking be subjective or objective? This was (and probably is still) a crucial debate in the realm of Web search. The issue is not only a technological argument but also a business decision.

Technologically, in the time before Google the performance of subjective ranking was generally incomparable to the performance of objective ranking. Yahoo was providing significantly better relevancy in its search results than the other search engines which performed objective ranking methods. Although due to the subjective policy Yahoo's execution expense was higher, its superior quality of search results made the cost worthy.

On the side of business execution, from the beginning Web search was coupled with the online advertising business model. Popularly, Web search engines were selling their first few related search results to certain advertisers; such a policy was once the basis of online advertisement. The subjective ranking search engines apparently executed the policy better than their objective competitors because the former ones selected relevancy subjectively anyway. There was nearly no extra cost for subjective search engines to embed the business model into their framework. On the contrary, the objective search engines might have to execute two policies in parallel to reach both the technological goal and the business goal. Therefore, the overhead of subjectively deploying advertisement reduced the advantage of objective ranking in its low cost of execution.

Google cleverly resolved the dilemma on the side of objective ranking. As the result, objective ranking declared the victory over subjective ranking, at least until the present. (Be note that now the subjective search strategy is striking back. Represented by Mahalo, the subjective search policy is reclaiming its momentum. I will analyze this phenomenon in the following installment of this series when discussing the challenge Google faces.) Another consequence of the resolution is what we all know: the victory clinched Google's championship on Web search over the previous leader Yahoo.

What Google did was actually on two folds. One fold was at the technological side on which Google implemented a brand new objective ranking policy that significantly improved the performance. We will discuss this fold a little bit later in this post.

The other fold was the revision of the online advertising business model. We have known that the previous online advertising business model favored subjective search. Since Google's technology belonged to objective search, the company was trying to discover a new business model that was more compatible to the objective search. Finally, Google invented AdSense and its sister program AdWords. There have been many discussions about the two programs. In fact, at the next installment I will discuss the two programs again. At here, however, the discussion is solely on the impact of the two programs to the combat between the subjective search and the objective search.

AdSense and AdWords reshaped the online advertising business model from the subjective judgments made by the search engines to the objective decisions made by the run-time mapping between search queries and advertising words. Please be note that this change did not indeed have improved the performance of advertising in the sense of technology. It, however, brought two critical improvements for online advertising business: (1) it decreases the unit cost of deploying an advertising word, hence by spending the same amount of money advertisers may now ask for more advertising words associated to their advertisement; and (2) it is implicitly empowered by the effect of the long tail since objective search automatically associates any unpopular deployment of the advertising words to the advertisement.

In order to thoroughly exploring the advantage of objective ranking, Google made another great decision. Google abandoned mixing the search results with advertisers' product links. Google displays all search results in order purely by their objective ranks. By contrast, advertisers' links are laid separately in such as the side bar of the page of the related search results. By this revision of advertisement deployment, Google won the name of integrity and objectiveness on Web search. This fame has been a crucial part of the foundation for Google's business success.

Cluster weighting vs. Link popularity

Another debate of Web search is between ranking by cluster weighting and ranking by link popularity.

In short, ranking by cluster weighting is to classify documents based on the measurement between typed keywords and predefined clusters of information. The implementation of the methodology may be machine learning or the mathematical analysis of vector computation. Ranking by link popularity, however, is to measure the relevancy of documents based on how popular the document is linked in the Web. More popularly linked ones are assigned greater value of relevancy.

Certainly, as many of us know, Google is a supporter of the latter policy. The PageRank algorithm is a famous representative of the thought.

In theory, however, we should expect that cluster weighting must be superior to link popularity on ranking search results. Essentially, cluster weighting methodology directly looks for the content relevancy, while link popularity just indirectly reveals the favorite of relevancy through human activities. In the other words, the truth itself is always more correct than people's vote of the truth. If we can directly reveal the truth, we do not need to rely on the secondary votes to guess the truth.

The problem is, however, whether we are truly able to compute the truth or even if we could, how much the computation would cost. This is where the problem of ranking by cluster weighting is.

The beauty of PageRank is to greatly avoid the complexity of vector computation between search keywords and content keywords. The mentioned computation is very costly in both of run-time execution and off-time optimization requirement. By investing the same amount of money to store and analyze the topological structure of World Wide Web, Google believes that it might gain more reward than investing on cluster weighting computation. Indeed, Google proves itself.

In essence, public voting is probably the most cost-efficient way to approach the truth when the truth is unknown. Public voting does not always reveals the truth. But if the truth is too expensive to be revealed, public voting is what most of the people accept and it generally reveals something that is close to the truth. This is the same philosophy of democracy. So actually Google was excising Web 2.0 implicitly even before Web 2.0. In person, I believe this is an important reason that Google eventually became an early leader of Web 2.0.

But ranking by cluster weighting is not dying. The policy is actually preparing its strong fightback now. Semantic search is the modern mutation of this traditional policy. As many people suggest and expect, semantic search (if it be realized) would certainly outperform the current search executed by Google. But how to reduce the execution cost remains to be a grand challenge. We will discuss more about semantic search in the following installments.

Open Sesame

In summary, Google's "open sesame" is to explicitly execute low-cost objective ranking over implicit, free, subjective ranking (link popularity) performed by regular Web users. This is a model putting "objectiveness" on top of "subjectiveness of the crowd."

However, the business success of this "open sesame" was still not enough for Google to be an age. Google might have been just another successful company if nothing else happened. There is another crucial reason that eventually pushed Google from a successful company to a legend. At the next installment we are going to discuss it.

(The next installment: Web 2.0)

Referenced resources:

• AdSense (Wikipedia)
• AdWords (Wikipedia)
• The Long Tail (Wikipedia)
• Page Rank (Wikipedia)
  

YOU: Time Magazine's Person of the Year, and LONG TAIL

The increasing influence of Web 2.0 has permeated the spirit of Long Tail into every aspect of human life. Probably Web 2.0 itself is nothing but the demonstration of Long Tail: an aggregation of nobodies eventually becomes somebody. With this philosophy, the person of the Year 2006 (as elected by the Time Magazine) becomes reasonably understandable. Although YOU are (most likely) a nobody by yourself, YOU can be part of a somebody through the effect of Web 2.0. Therefore, every one of YOU is the person of 2006 because YOU are somebody!

Web 2.0 is changing the world. Long Tail demonstrates its power. In history we have never been so close to the real power of collectivism.

The Scottish philosopher Thomas Carlyle once said that the history was about "Great Man." But every "Great Man" came out of dozens or hundreds of "less greater man," who again came out of thousands or millions of nobodies who were long tails. We often overlook the impact of these long tails because the impact from every one of them is so tiny comparing to the power brought by a "great men."

Fortunately, the rise of Web 2.0 changes the history. The beautiful Web 2.0 allows everybody to have a chance being a "Great Man" through engaging the effect of Long Tail about many others. The great diversity of the Web brings the world tremendous opportunities. Through Long Tail, Web 2.0 constitutes a real fantasy that "I work for everybody; and everybody works for me." Or it may be restated in a more fashion style: I am in your long tail; and you are at my long tail.

So, the person of the year 2006 might better be YOU with a Long Tail!

Referenced resources:

• Long Tail