Bing blogs

This is a place devoted to giving you deeper insight
into the news, trends, people and technology behind Bing.

Search Blog

May
22

Opting Out of Open Directory Listings for Webmasters

Here in Search, we are always interested in hearing about ways to improve the search experience. And, along with Danny Sullivan and Dave Winer, customers have let us know that they wanted us to change how we used Open Directory descriptions in search results. So… we did!

Just to give some background, the Open Directory Project at dmoz.org is a repository of millions of human-edited descriptions. Even though these human-edited descriptions provide a lot of value, with human editing may come human error, bias, descriptions getting outdated, or the editor’s text may simply not suit the webmasters who want to be represented in their own way.

What has bothered the webmasters previously is that when search engines preferred search result descriptions from dmoz.org, they did not empower webmasters to opt-out of those descriptions. This can be especially annoying if the descriptions from dmoz.org are outdated, or just plain inaccurate.

We had one customer who was frustrated because the ODP description of their site mentioned “favours” and was listed under Canada when their site was actually in the United States and was spelled as “favors”. All they wanted was a way to specify that MSN Search should use the description from their page instead of using ODP.

So what we did was introduce a new option at the page level  - a robots meta tag – that tells the MSN search bot not to use the DMOZ site snippet.  This is something that only can be done at Web page level, by a webmaster, and is not done as part of the robot.txt file.

So in your Web page you’d put

 

<META NAME="ROBOTS" CONTENT="NOODP">

 

or

 

<META NAME="msnbot" CONTENT="NOODP">

In theory the first of these applies to all crawlers and the second just to us. As far as we know right now, we are the only search engine  to support this  tag, so the two are the same for the moment. But when others follow suit, you could use the second tag to get only MSN to ignore ODP content for your page.
A word of caution: Putting either tag in your pages will not make your search results descriptions change immediately – they will change once our crawler has re-crawled the page. Usually that takes about 1 day -4 weeks for us to re-crawl you (ok, that sounds odd, but we hope you know what we mean).  :)  

Try it out, and give us your feedback!

----Girish Kumar, MSN Search Development Lead

Betsynote May 30th: this post has been reformatted to correct some font weirdness. None of the content has changed.  

Comments

  • Great idea, thank you.  Hopefully, the other major search engines follow suit so that this tag is globally recognized.  The DMOZ has outlived its usefulness IMO.  
  • This is a practical idea - it levels the playing field.

    Often DMOZ and Yahoo would use very concise descriptions....SEOs wanted to make their descriptions more compelling on the SERPs.
  • "or the editor’s text may simply not suit the webmasters who want to be represented in their own way." which means "their own spammy seo way"
  • Why don't MSN and Google display DMOZ attribution on their SERP pages that use DMOZ data?
  • Great idea! Some time ago, we had a heated discussion at the Resource Zone with a webmaster who objected to the fact that the ODP description was used by, eh, another search engine :)

    If the major SE implement this tag, ODP may well get fewer "marketing driven" update requests...
  • MSN, great idea! @ first I loved DMOZ but then, when I saw that the description they gave to my website http://www.stilxxi.ro was not entirely accurate and that I can't change that, I began to fear the dmoz. It's almost like this: if you make a change, whether it's a small change or a big change, on the website, don't rely on dmoz: they'll never sense this.
    The human effort to create smth like dmoz is quite big but, as it has been said: we are human, we err! So, that's a pretty neat stuff about this tag!
  • Subsidiary question : how many webmasters wish that search engines use the DMOZ description to describe their pages : 1%, 2%,... ?

    This percentage is such a minority that it would be more useful to create this other version:

    <META NAME="ROBOTS" CONTENT="ODP">

    It would make it possible to ask search engines to use the DMOZ description when the webmaster believes DMOZ is better in describing the site than himself... ;o)

    Jean-Luc
  • I'm not sure about that.

    On the one hand it gives you more control over your listing in the SERPs. On the other hand... it gives webmasters more control over their listings in the SERPs.

    And if you're not happy with the description associated with your website in the ODP, you can always suggest a new (updated) description...
  • Good that MSN tries this way!

    DMOZ descriptions are not always true reflection of what the actual site is.

    Keep Going MSN...
  • I have many websites that even after one year didn't get included in OPD just out of lack of editors getting to them.  I wish I had the OPTION to 'dis them!!  I've looked long and hard for a work around for OPD....  
  • It is true that it takes a fairly long time to get listed on dmoz, and its good to know that MSN is protecting webmasters in a way from this disadvantage.. cheers..
  • Great move! I hope the other search engines catch the vision and follow suit.
  • DMOZ should be de-valued by Yahoo and Google as well..too much editors are corrupt selling
    links, using them to make money, some editors won't placer you on the pages where they have clients..they have to go!

    didn't they mention they want quality sites?
    and you check and plenty of those are of poor quality just for the purhose of selling links..
    I know of a couple editors sellings links to the public...did report them twice in two years and nothing has happened..lol
  • you're lovely :)
  • This is fantastic. Thanks.