Bing blogs

This is a place devoted to giving you deeper insight
into the news, trends, people and technology behind Bing.

Search Blog

January
18

Working Together Against Blog Spam

Our job is to provide the most relevant and useful results to a user’s search query.

Sounds easy.  But, unfortunately there’s a powerful financial incentive for a small subset of people to try to manipulate the results so their sites come up to the top – at the expense of our customers.

Fighting this web spam is one of the harder parts of our job.  Spammers are very creative people who evolve their techniques over time.  Over the past year, being the newbie to the search engine block, we have had to discover and combat all of the techniques that have been developed over the past 5 years.  Now, finally, we get to move forward!

One thorny source of spam that we’ve seen is people stuffing blogs with comments that include links back to their sites.  We have taken some steps to combat this – but we really wanted to find a way to put bloggers back in control.

I was excited to wake up this morning to an email from my long-time friend and college roommate who is currently an engineer working on search at Google.  Don’t worry Paul – I won’t blog any outrageous stories about our time at Princeton. :-)

Paul told me that Google is planning on announcing support for a <rel=”nofollow”> tag on individual <A> links.  Any link with this tag will indicate to a crawler it is not necessarily approved by this page and shouldn’t be followed nor contribute weight for ranking. Our Search Champs suggested this and it has been a part of our plans since, we think it’s a great idea.

I quickly circulated this around our hallway and also to the MSN Spaces team – and we got quick agreement from both teams.  Over the coming weeks, our MSNBot crawler will start respecting this new tag, and sometime after that MSN Spaces will start to support this as well (I expect Mike Torres and MC will have more details – the team has already made investments to help prevent blog spam to begin with). Cheers to Yahoo! and Six Apart for also supporting this movement.

This should be a good thing for search customers, bloggers and webmasters. Let us know if you agree (or not) either in our comments or in our wiki. We’ll be listening and learning.
 

Ken Moss
General Manager, MSN Search Dev & Test

 

Comments

  • I do agree with this new option for website owners.

    I have yet to see how much it will help keep blog comment SPAM down but if everyone begins implementing it on their blogs it should have some positive impact.

    The other reason I like this feature is that website owners have more control over what search engines do when visiting their website.

    Good job guys!
  • Thank you for setting this up.

    But how do folks like us who blog on dotnetjunkies take advantage of this? I can't be expected to add rel=nofollow in each link in the comments :-/
  • Remember that spammers are interested on people, not in search engines (i.e: mail spam).

    Regards
  • Thanks for willing to work together with your competition.
  • Hey you know something, with all the high profile links this page will be getting, this page will be a prime target for spammers. Can the feature be implemented on this page first :)

    Is this the starting point for search engine standards?
  • Nice idea; but open to misapplication by those wishing to "horde PR" [for Google thoughts] or as a nifty way to "share links" but, at the time, having the search engines "ignore" those voluntarily shared links by the webmaster. Only thing I can say it is easier than the javascripted methods oft-advised as a way to obfuscate links to try not having search engine spiders follow those or credit them being shared at all.

    *sigh* One step forward for the blog, wiki, and/or guestbook owners but, in some ways, two steps back for those of us not one iota interested in sharing our URLs in those setings or rigging up linking exchange/strategies purely to impress search engines. Why? People are already sharing ideas of how to misapply this attribute to links, and comment spam is not always the reason shared, as a nifty and easy way to devalue links shared in the eyes of Yahoo, MSN, and Google.
  • Yay. The rumours are true. Comment spam has been a huge hassle so it's great to see you teaming up with Google on the rel=nofollow tag. As an ethical seo/writer gal I really appreciate it. I'm always amazed at how quickly things like this can happen in the web industry, especially when they benefit everyone. I'm sure some folks will subvert the tag for evil - but hey - we'll find a way around that as well when it happens. Cheers!

    -- Search Gal
  • this isnt going to help stopping automated spam bots from posting links, but hopefully it will deterr the humans that are doing it the long way.
  • Please update the post to use more appropriate terminology. It is not a tag. It is a new attribute value for the REL attribute. Nothing more. And for your information, REL applies to the LINK element as well.
  • <a href="http://adamjh.blogspot.com/2004/12/blog-post.html">I">http://adamjh.blogspot.com/2004/12/blog-post.html">I suggested this idea on my blog</a> about a month and a half ago. It's awesome that Google, Yahoo, and MSN were able to work together to turn it into reality.

    The potential goes far beyond curbing blog spam. Perhaps more importantly, it provides a means for publishers to link to information without lending creed to it.

    One of the most prominent example of a reason publishers would want to do this are educational sites that link to sites such as www.martinlutherking.org in order to teach children about misinformation on the Internet. In doing so, they inadvertantly raise its search engine rankings, propogating the misinformation itself.

    Check out the blog post if you want to read more: http://adamjh.blogspot.com/2004/12/blog-post.html
  • Has anyone ever thought about the uninteded possibility which this feauture makes possible?
    Lots of webmaster who falsly think if they link other sites they give away some of their ranking and therefor use meta refreshes (which cause heavy bugs (url hijacking) in google) now just use the rel='nofollow' tag to mark all their external links. It may be a good thing for the litte part of the blog spam links, but I don't think everybody has thought about alle the possibilities...
    Is this really a solution? It takes spammers Rankings, but they still got the links... think about that!
    (excuse my english, I'm not a native... ;-)
  • This is just pitifully stupid. This is being purported as a way to stop spammers from submitting "Comment spam.", right? Do you really think that comment spammers, the same people who send out spam emails, care if 2% of the sites add the "rel" tag? No! They will just keep on submitting the comment spam, and maybe more of it if their search engine ranking start teetering. The right way to do this is to PAY people to FIND GOOD WEBSITES and then only index those for POPULAR QUERIES. Of course, MSN cares not about quality search results, they care about CPM, money and stealing market share from google and yahoo. This is obvious from the fact that they hemmed and hawed, and eventually dumped their editorial search product.

    Then let's think about PageWeight/Rank/ or whatever. This is supposed to be a mathematical formula, and it has certain constraints. One of those is that the formula must somewhat reflect the way a user moves thru the web. All the link spammers have created a non-ideal situation where some sites have many links to them that no real person actually follows.

    Well now you've made the problem twice as bad because all the webmasters and spam sites out there can now use this special tag to further influence how the webgraph is created. If you want a site with lots of page weight, you make all your links "nofollow" and that way all the page weight of anyone linking to your site Stays in your site. Unfortunately, this totally breaks the page weight calculation, because all of those good links are now destroyed.

    Duh!
  • Jason,

    I don't think your argument is entirely valid. Linking to another site does not strip away ranking of your own page it only helps to raise the pages that you link to. Using the no follow tag will only influence the credibility of the page that you are linking to, it will not affect at all the value of the page that you are on.
  • I believe that this decision needs much more thinking out.
    How will the attribute "no follow" exactly work? I mean why should bloggers and other web masters be in control of how search engines rate pages. In this way, products or companies' web pages that are hated by a vocal group or the Internet community in general might get lower rankings just because people have decided to put "no follow" in front of all their links. One of these companies might be Microsoft!
    In addition and most importantly how will each search engine interprete the "no follow". Is there an agreement on this?
    And what about practical issues: fighting spam with "no follow" means that
    1. the person writing the blog will have to review every comment and flag those that are spam with "no follow. This is impractical. Why doesn't he delete the spam comments instead? Or,
    2. the blogger system itself will mark all comments with the "no follow" automatically potentially lowering the rankings of comments which link to pages that are not spam and whose ranking should not be lowered, since they might refer to a competitor or to the opposite opinion to the one written on that blog. Or,
    3. the blogger system can automatically use a kind of smart technology to mark those comments which are considered as spam with the "no follow". Why not delete them or why not place them in a special folder instead?
    I think that "no follow" might be a good idea or it might not be depending on how it is used or abused by users. However, users should not be the ones deciding the ranking of Internet pages but rankings should be decided mostly on relevance of content. Also, nothing will prevent web masters from tampering with the popularity of a web page by using the power of "no follow".
    Please think about this a bit more before implementing it.
  • You say that bloggers should be in control. Well, I think not. Justice to all web pages should be in control.
    The problem with blog spam is the following:
    Links on ordinary web pages are there because they are important for the functioning or the friendliness of that web page. Thus, if many links to a page are found this means that the page in question has been chosen by many other Internet users and it should be relevant to a search query because of its popularity. Well no. Are the most famous politicians the best? Popularity can be missleading and so I was hoping that more important things were taken into account in order not to keep "unpopular" pages always hidden from us and thus always unpopular. In a sense the more popular get more popular and the unpopular stay unpopular.
    On blogs things are different. Pages can be made popular by their owner by entering comments on "competing" blogs. Popularity in this respect might be more related to the actual relevance. Blogs are much different than ordinary web pages. They are a community. They should be treated like this by search engines. devising an attribute is not a solution. There should be a different system for filtering web pagess based on their content and not on the number links present pointing to them. There should be a different search algorithm especially for the blogs. Blogs are a different kind of web.