Here’s an interview with Google’s vp of engineering about search engine quality.
“we have a wonderful team of people whose sole purpose is to evaluate search results, that’s what they do. They use hundreds and thousands of different data points and they measure things all the time, and they also evaluate those experiments and those new ideas.”
“PageRank is still an important parameter, but it’s just one parameter. And, there are all kinds of parameters, [...] all the obvious traditional information retrieval parameters. There are many others that we invented and there is the combination of all of them, which is really where the hard work is being done…”
“We don’t change things with specific queries. We don’t improve one query at a time. That does not scale.”
“Q: Are there other examples you can provide or anecdotes of situations that led to algorithm tweaks or improvements? [Palo Alto example in the New York Times] Udi Manber: There are many such examples, but I’m, I am hesitant to go into specific ones, because, that’s our big advantage.”
“I think, personalization is very important, but I think over-personalization is probably not good. The key is to find the right balance. If we discover that you’re interested in sports, and then you search for medical information, we shouldn’t give you only sports medicine results.”
“If you search for “I have a dream”, if you did that five years ago, you were probably looking for the text of the speech. You wouldn’t think necessarily that you could actually see that speech. So, you wouldn’t search for the video of the speech because you wouldn’t think that you can do that. And, if we show it to you, it’s not necessarily what you said you are looking for it, but there is a good chance that you want to do that. That’s a nice discovery, and will cause people to search more videos, because now they realize that they can find more videos.”
“Q: Do you picture at anytime having any kind of editorial oversight in the form of Google employees looking over results an editorial perspective? Udi Manber: We have no plans of doing that in the foreseeable future.“