Bing blogs

This is a place devoted to giving you deeper insight
into the news, trends, people and technology behind Bing.

Search Blog


Thoughts on search quality

This morning, I will be on a panel at the Farsight Summit with some of the industry’s thought leaders to talk about search quality as we look into the future of search. Farsight is about looking 10 years into the future to explore the big industry challenges and opportunities ahead of us as we work to deliver the next generation of search.

We woke up to an interesting (and interestingly timed) article by Danny Sullivan about some complaints Google has about how it says Bing ranks results. I wanted to take a moment to make a couple of points in advance of this panel so we can stay focused on the original intent of the Summit.

The Bing engineering team has been working hard over the past couple of years to deliver the best search relevance and quality in the industry and for our users. This is our top priority every day.

We use over 1,000 different signals and features in our ranking algorithm. A small piece of that is clickstream data we get from some of our customers, who opt-in to sharing anonymous data as they navigate the web in order to help us improve the experience for all users.

To be clear, we learn from all of our customers. What we saw in today’s story was a spy-novelesque stunt to generate extreme outliers in tail query ranking. It was a creative tactic by a competitor, and we’ll take it as a back-handed compliment. But it doesn’t accurately portray how we use opt-in customer data as one of many inputs to help improve our user experience.

The history of the web and the improvement of a broad array of consumer and business experiences is actually the story of collective intelligence, from sharing HTML documents to hypertext links to click data and beyond. Many companies across the Internet use this collective intelligence to make their products better every day.

We all learn from our collective customers, and we all should.

From its inception, we have had what we believe is a distinct approach to search, and the features and innovation in Bing – from our new user experience and visual organization approach to our focus on inferring user intent and helping customers complete complex tasks, Bing has added a new voice and new experiences to search. We never set out to build another version of an existing search engine. We believe search needs to do more for customers. This is the guiding principle in how we approach our work each day and it is why we are here today engaging in discussions at Farsight, an event we are proud to have helped bring about.

This is why I am excited to mix it up this morning with some of the best thinkers in the search business.

We all have lots of work to do to get search to be everything we want it to be, and everything our customers deserve. Let’s get to it.

Harry Shum, PhD, Corporate Vice President, Bing


  • I have to say, this entire fiasco is destroying Bing's reputation, I'm having relatives e-mail me about this just to take jabs at Bing, because I told them that I really like Bing.  People now think that Bing is basically copying all of Google's results.

  • Harry - I'm glad to see Bing being transparent about data usage and honestly, I think this is the right call. When a market leader has a 7-year headstart, I fully expect that you should use every arsenal in the toolbox.

    This is, to some degree, within Google's rights to complain about. You're using their data in a smart way to improve relevancy and they built a nice honeytrap to expose it. That said, the timing of their release made them look really bad. To sit on info like this for 30 days, waiting until they knew you'd be in the room together and employing the news to deflect from their own spam/relevancy issues just isn't cool.

    Here's to hoping that we can be more professional and authentic about criticism, praise and search quality issues in the future on both sides.

  • So, you basically admitted you're spying on Google Search to steal their quality results.

    It's not surprising, they proved it, so you cannot say you didn't do that.


  • @randfish

    Why would someone put an arsenal in a toolbox? You really need to work on your metaphors. As far as shifting blame goes, you got that one down.

  • In 1000 different signals you have you've picked the worst one. Direct copying of results is just below anything.

    @randfish: yes, MSN Search/Live/Bing had 7-year start as a market leader (actually, I thought it was AltaVista?) but now they aren't. Now we have only what is, essentially, a Google carbon copy, an imitator, an imposter. Sad.

  • @asdasd

    But it's not a google carbon copy, like the post said, using user habits to index is only one way that it indexes.  It doesn't use the Google Search results to say, "Ok, so this is on Google's front page, I should now put this on Bing's front page".  It simply indexes the site.

  • Well played Bing team :)

  • Those who think this is "direct copying of results" are mistaken. Google's spinning the PR that way, but that's not what happens. Bing uses signals like clickthrough on SERPs in their 1000 factor algorithm. When you remove any other signals (which is what Google's honeypot did), all you're left with is the 1 Google thought they catch MS using, and that's what happened.

    I know it's compelling to attach onto the simple "Bing stole from Google" story, but like everything else in the media, it's a vast oversimplification. I'm hard-pressed to believe that anyone who was tasked with beating Google at search wouldn't employ this tactic if they had access to it.

  • "But it doesn’t accurately portray how we use opt-in customer data as one of many inputs to help improve our user experience."

    You mean how Bing uses IE/toolbar user's personal communication between the user and a 3rd party website? which you opt-in for them by default?  Yeah, your portraying of the story is more accurate alright.

    If every arsenal in your toolbox is fair game, then why don't you just redirect all "" url to "" in IE while you are at it.

  • So when are we going to start seeing "Powered by Google" on Bing results?

  • Many companies have internal database-based search for the products on their catalog.  So if a customer is using my own search and locates a product in my catalog, and happens to have the Bing toolbar installed, then Bing Toolbar will pass that information to Microsoft.

    Then next time someone asks Bing about that product, Bing will be able to direct them directly to my product page.

    So suddenly OMG Bing is copying my search algorithm?  And this practice should stop?

    Personally I really don't see what's wrong with this.  In fact, if the data helps Microsoft identify why I couldn't find something in Bing but I could with Google, so they can tweak their query and improve the results, I'm all for it.

    The whole "copying" is a really lousy marketing spin.


    When is the last time IE automatically installed the toolbar or opt-in by default?  Have you actually done this recently or are you just prejudiced to begin with?

  • So if I took years of R&D to make the algorithm that can find your result, and the other company can just use my result and display as their own?  I don't think so.  If Bing can't find something for you.  They should just improve their algorithm so they can find the result.  Get result from others is low.

  • From the orignal article by Danny Sullivan, linked above.

    "When you install the toolbar, by default it is set to collect information to 'improve' your experience, as you can see:"

  • A more accurate discription of this approach might be that Bing is using the click behavior of google users to improve Bing results - not google results. There is an important distinction in that statement. Bing is not copying Google or their results -rather It is learning from user behaviors across several sites, one of which might include google. Another example might be if lots of people are visiting a breaking news article, one could imagine Bing might consider ranking that article higher in their news section based on user click behavior/interest. Yet, that would mean that Bing is copying the ranking of news sites, who might also have that result at a top spot on their homepage.

  • I wonder how you'd feel if your child got caught cheating in a test at school and told you "but I only copied a couple of the answers". Microsoft's behavior is outright cheating, copying the other kids results, arguing about the degree of cheating just makes you look like a guilty schoolchild in front of the headmaster.