Sunday, May 25, 2008

The Series of Web Evolution, revised

I have just finished a total revision of my series of Web evolution in Thinking Space. The changes include many internal rewritings and even the change of titles of several installments. If you are interested in it, at the following I have attached the links to all the ten installments.

Moreover, this revision is compatible to my new feedback about the Harmonious age that I would post soon in Thinking Space. Be together, I think the two parts composes a complete vision about the new Web from both of the philosophical view and the technological view. The next question is how to engage them into the real-world development. I would be glad to discuss with anybody who is willing to explore the path towards the next generation Web (commonly known to be Web 3.0). Now it is the timing to start up real Web-3.0 solutions by my judgment.

If you are interested in discussion, please either leave a comment with your contact information in this blog or feel free dropping me an email at: yihongd@gmail.com.

A View of Web Evolution

1. In the Beginning …
2. Three Evolutionary Elements
3. Two Postulates
4. Web Evolution and Human Growth
5. Evolutionary Stage
6. Qualities of Evolutionary Stages
7. Trigger of Transition
8. Beginning of a Stage Transition
9. Essence of Web Evolution
10. Completion of a Stage Transition

Wednesday, May 21, 2008

Pay you to Live Search, brilliant?

Both TechCrunch and VentureBeat reported that Microsoft would announce a new search advertising model, which pays users who use the Live Search engine to search and eventually finish an online transaction. Is this a brilliant idea? Or not?

One week ago, I was at Redmond with the Live Search team. In Live Search, there is a group whose job is "to explore all the crazy ideas." The group picked me to interview and asked about my "crazy ideas". Unfortunately, however, I did not have any crazy idea except of a fairly novel vision suggested for Microsoft to compete with Google. In short, my suggestion was a totally un-Google Web search strategy that the Live Search crazy-idea people could not catch where my craziness was. As the result, they were not impressed by idea that did not sound crazy. Now I see what the Live-Search-craziness is.

The philosophy beneath this "crazy" idea is straightforward: when there are two sites from which we could buy the same product, we often choose the site that gives me more discount when checking out. Since Microsoft has lots of money, why not directly buy searchers from Google?

Is the idea "crazy"? Crazy, indeed.

I would like to quote a comment written by a reader (Tyler Wright) of TechCrunch. He has made a very cute analogy that points to the problem of this "crazy" idea---

"GM and Ford offer cash back to buyers, and they’re on their way out - and losing credibility daily.. Sounds kinda similar."

Ah-ha, this is the problem. GM and Ford often do more discount and promotion than Honda or Toyota does. But the discount and promotion seem do not really save their fate. Why does Microsoft believe that discount and promotion would save it from Google?

A deeper thought behind this strategy is that "online search" and "online transaction" are actually two varied phases. At the online-search phase, we look for a good search engine that can help us find what we want quickly and conveniently. During this phase, we also frequently look for advertisement provided by the search engine. By contrast, at the online-transaction phase, we simply want to finish the transaction as quick as possible. But at the same time, an extra bargain is alway appreciated. Very few people, however, will continue looking for product advertisement when they are checking out (because they are tired of long-time shopping).

By the former analysis, I cannot see why I should abandon Google for being my search engine, while at the same time I can use Live Search to check out. If many Web users adopt this simple strategy to maximize their benefits, I cannot see how much Live Search may gain by this scenario. To the end, Live Search could gain a few net-flow from Google due to the final check-out transaction. The problem is, however, this extra net-flow gained by Microsoft does little help for prompting advertisement in Live Search. This is thus the key of the entire issue.

Sorry, I am still too calm to be crazy.

Monday, May 19, 2008

The Harmonious Age?

The Harmonious Age is an interesting philosophical view written by Adam Lindemann. Adam is a liquid thinker whose thoughts are compelling while at the same time he is also a clear-mind businessman who is very practical in action. It is hard to find the two great characters within one person simultaneously.

I am particularly blogging this thought here because the derivation of this thought fits well to my philosophical view of Web evolution. In the later posts, I will explain the similarity in more details and especially on explaining how Adam's thought means in a broader picture of the new industrial revolution we are experiencing at present. But in this post, I just want to first introduce what he think that "we are moving through the transitional period between the Industrial Capitalistic Age and a new age that might be described as the Harmonious Age."

One more word before proceeding. It is interesting to notice the similarity between the Harmonious Age used by Lindemann and the Harmonious Society slogan named by the current Chinese President Hu Jintao and Premier Wen Jiabao, though I am quite certain that there is no necessary link between the two. The theory of Harmonious Society has been the foundation of the so-called Hu-Wen New Administration that leads to the current economical boom in China. With this successful comparison, could we expect Harmonious Age be another great theory to lead us to a new Web age?

The following is written by Lindemann. My interpretation will be posted in later posts. Please keep on watching if you are also interested in it. Trust me, it could be more interesting than you thought.


Before the 1800s there was a feudal society where wealth was stored in land and real estate.

After the 1800s and the industrial revolution wealth began to be created on a massive industrial scale. In this industrial capitalistic society, wealth was stored in a financial system that created many different types of money.

Real money such as cash, stocks and shares and bonds. It was the mind of man that harnessed the awesome laws of nature as well as its energy and resources to fuel this industrial capitalistic society.

However, mankind is reaching a limit to its ability to harness the natural physical resources of the earth such as oil, gas and other fossil fuels and hence growth is becoming unsustainable. The fundamental problem is that man based his survival at odds with an not in harmony with the natural renewable balance of nature. The mind of man did not truly understand the symphony of live on earth - it tried and succeeded somewhat in being clever and failed to be wise. This was a time of division in knowledge and hearts and minds. It was a time of artificial abundance for some and fear and scarcity for others. It was a time of nation states and world wars.

The 21st century will be an age where man achieves wisdom and learns to work in harmony with the natural physical and spiritual laws of the universe such that we live in balance and harmony with the earth and each other. It will be a time of the fusion of knowledge, hearts and minds. It will be a time where the illusion of scarcity will become replaced by the reality of abundance. It will be a time of peace and sustainable life for humanity on earth.

The key to understanding this shift is to realise that its all a mindset. Money=Energy and Energy = states of matter. The feudal society can be seen a solid. The industrial capitalistic society can be seen as liquid and the new age of the 21st century and beyond can be seen as gas.

Saturday, May 17, 2008

Building Semantics is different from Building the Web

In my newest post at Semantic Focus, I described the difference between building semantics and building the Web. A real Semantic-Web company needs to demonstrate its ability on both aspects instead of only one of them. By simply adopting RDF, a company cannot automatically become a Semantic-Web company.

Thursday, May 15, 2008

Si Chuan Earthquake, People need help!

As you may know, Thinking Space is a place we discuss World Wide Web related topics. Very rarely I post other topics beyond this theme. But this time, I have to do so for the recent massive 7.9-magnitude earthquake at SiChuan province, China. Tens of thousands people have been dead and more are injured. These people need help, and please help them as you can.

The following is a slide show made by my friend Oliver Ding.


Please Help Earthquake Victims In China


From: OliverDing, 8 hours ago





As you may have known, a massive 7.9-magnitude earthquake rocked central China's Wenchuan County at around 14:00 5/12/08. The death toll was more than 19,500 by 16: 00 May 15th Beijing time, and much more were injured. The figures still continue to rise:
http://www.msnbc.msn.com/id/24589987/
Many of the victims were children!! Children died when they are in school! Many of them are the only child in a family! The entire Chinese nation is fighting with the disaster!
We need help from the world!
Please find the kindness in your heart and help whatever you can!


SlideShare Link

Monday, May 12, 2008

How to achieve the Semantic Web

The realization of Semantic Web becomes a chick-and-egg dilemma. Without Semantic Web applications, we can hardly build Semantic Web data. But without Semantic Web data, who are going to implement the useless Semantic Web applications? If Semantic Web really represents the future of the Web, we must have overlooked something crucial.

In a recent post at Internet Evolution, I emphasized again that the missing link is the role of humans in the Web. Semantic Web has been thought by many researchers to be equivalent to a web of data. This is thus the problem. Semantic Web is a web of data. But a sole web of data is not sufficient to be Semantic Web. They are not equivalent. The problem is that in the expression of "a web of data" humans disappear. It is this miss that causes the dilemma of Semantic Web.

This connection between WWW and humans seems has been puzzled by many people, including even top researchers. Last Friday, I had tried to ask this relation to a principal researcher at Microsoft and he did not give me a satisfactory answer either. I bet, however, that this connection is quite straightforward to many Web-2.0 people. Again, we are sorry to see the big gap between the industrial Web 2.0 and academic Semantic Web.

More details of my arguments can be watched in the original article. As usual, many thanks to James Johnson who helped edit my post.

Sunday, May 11, 2008

Return from Microsoft

The past weekend I attended a special event organized by the Microsoft Live Search team. The Live Search team invited 28 up-to-graduate PhDs from US and Canada to come to an on-site interview. The uniqueness of this event is that nearly none of the 28 candidates have been telephone interviewed by the team before and many candidates (including me) even have not submitted a resume to Microsoft before being invited. The Live Search team searched out these candidates (probably through the Live Search) and invited us. The theme over the event is straight and clear---beat Google!

About the Trip

In general, the trip is full of pleasant. The Microsoft recruiting team have organized a wonderful event. It is warm, joyful, and with exciting surprises. Thank you, Erin Bucholz (my direct contact recruiter), Jared Singer, Rondell Honcoop, Ben Mercer, and others (sorry I am not good at memorizing names).

Each of the 28 candidates is a selected one with a particular background that is related to Web search. For instance, there are candidates with the background of query optimization, distributed computing, image processing, data mining, natural language processing, and so on. My particular background is labeled Semantic Web. Moreover, I found that I was the only candidate invited to this event with the primary background as Semantic Web. It is exciting to be in a group of young scientists of varied disciplinary areas while at the same time with a focused general theme.

Microsoft has arranged most of the candidates (including me) staying at Westin Bellevue, a modern 5-star hotel that is adjacent to upscale shopping centers. The general environment of the hotel is superior. The guestrooms are luxuriously furnished in modern style---clean, simple, straight, and with modern decor.




At the very morning of the interview day, Microsoft sent a limousine (a 14-passenger Navigator model I believe) to pick up the candidates to the campus. It was a slight surprise to everybody and we were joking to attend a party rather than an interview event.

limousine exterior
limousine interior

The Interview Event

In this interview event, the Live Search team has scheduled three group discussions and four-round individual interview for every candidate. All the three group-discussion leaders are well-known Web researchers from Microsoft Research. They are Yi-Min Wang, Chris Burges, and Paul Viola. There are totally more than 10 individual interviewers. Each of them is a program leader with a particular focus on Live Search. Microsoft was indeed serious to this event.

Yi-Min Wang is a passionate speaker. He has a strong passion on competing Google. At the beginning of his session, Wang briefly introduced himself and described a general paradigm of Web search from the industrial point of view. In short, Web search is about finding a matching between billions of Web pages and billions of search queries, while at the same time the numbers of both pages and queries are increasing.

After the brief start, the rest of the session was focused on questioning and answering. In particular, when answering a question Wang described his experiences on fighting to fake Web pages, which was once reported by The New York Time. This topic also led to many discussions of a broader issue of the online advertisement business model and the risk of this popular business model.

I asked a question to Wang how he thought of the factor of humans in his described grand picture of Web search. His answer was primarily focused on the user interface issue. Nevertheless did I agree with his points, he had neglected mentioning the connections between Web resources (such as data, services, and links) and the humans who create or own these resources. I thought that the factor of these connections should at least be another critical issue with respect to my question. But time for question answering was limited and thus he might just not have enough time to expand his discussion.

The session with Burges was slightly different from the previous one. Burges began by asking every candidate why they came to this event. I answered by quoting myself a motto---"We may beat Google, by not by following the Google way." Microsoft is thinking of defeating Google on Web search. Hence I am very interested in coming to hear how Microsoft would approach this goal and I am willing to share with Microsoft how I think this goal could be approached. I would be glad to join them towards this goal together.

The addressing of "beating Google" does not mean at all, however, that Google is bad or evil. It is only about that we need competition to improve Web search better and better. Eventually Web users will be the biggest winners. With the same purpose, I have shared my viewpoints with the CEOs of Mechanical Zoo and Imindi (two ambitious startup companies but with great potential) in the last two weeks. Max Ventilla (Mechanical Zoo), Adam Lindemann (Imindi), and I have shared this common belief---by not following the Google way, we may approach alternative fascinating solutions for Web search and Web knowledge reorganization. Again, I shared this motto with Burges and he told me that it was also exactly what he believed.

Burges did a brief slide show for us about his new role at the Web search team and the Microsoft Live Search ranking algorithm. After his talk, I asked how he would compare the Microsoft ranking to the famous PageRank algorithm used by Google. He replied that the real ranking algorithm used by Google has already not been the original PageRank for long time. The very core of the Google ranking algorithm is a top secret. But Microsoft is catching up.

I agree with Burges. The PageRank algorithm is too raw for real-world products. To get high quality search results, Google must have done a significant revision of this general algorithm. The revision might have been so great that the eventual Google ranking used now may actually no longer be named the PageRank in its standard sense.

On the other hand, however, the PageRank algorithm reflects the Google's philosophical vision of World Wide Web. Google evaluates the Web to be a network of linked nodes where the importance of individual nodes is primarily determined by the linking popularity inside the entire network. This philosophy is the fundamental of "the Google way." Hence by "not following the Google way" it means that we need to think of the Web in a fairly different picture from the one Google thinks. I have such a different picture described. Adam Lindemann of Imindi shares a very similar vision as mine. But what is the picture that Microsoft thinks? Burges had not directly addressed an answer, and neither had Wang. Unfortunately, I have not gotten another chance to explore this issue deeper with another Microsoft developer or researcher in this trip.

In the third session, Paul Viola did a formal presentation to all the candidates in lunch. His talk was primarily on how to do a good research particularly in an industrial company rather than in an academic institute. He made several constructive suggestions for young scientists and engineers. The talk is informing and helpful. Due to the time constraint, however, we have no chances to ask him specific questions.

After the lunch, we came to the most fun part of the event---the swimsuit competition (i.e., individual interviews).

All the interviewers are excellent leading developers and researchers at Live Search. They are very well experienced and more important, they all have the passion on what they are doing (at least for the four who had interviewed me).

The four-round individual interview is scheduled into four main topics---research background discussion, coding, algorithm, and design issues. At each round, a candidate got to talk to an interviewer for about 45 minutes. In general, I have done a fair but not an excellent interview. I have described my thoughts to them, written a few lines of code, and solved some problems. At the same time, however, there was a problem I could not figure out the final answer to the end. I had tried to think of it from all angles except of the angle the interviewer looked for (and it is not a familiar territory of mine, what a pity).

In short, I had emphasized two points about the future of Web search: (1) the switch from the "God" role to the "Guru" role of search engines, and (2) the importance of "proactiveness" in the next-generation Web search. I emphasized to the last interviewer (a kind lady) that we were not just searching the Web. By contrast, we are searching the evolving Web; and this is a key when thinking of beating Google. If Live Search can think of the Web evolution a step further beyond Google, Live Search would get a better chance to beat Google.

Final Address

The final decision of this interview event will come to me in few days. No matter whatever, however, I appreciate this oppotunity and it gives me a chance to hear the first-hand opinions about the future of Web search from the most frontier industrial developers and researchers as well as from many peer PhD candidates all over the North America, let it alone that I had a wonderful journey at Seattle.