Another crawler in your logs

  • Comments (35)

Live Search is committed to continuing to improve how we discover and crawl your content. In the coming weeks, we will be testing an update to MSNBot, which may show up as a new crawler name in your referrer logs. The new crawler user agent string will appear as:

msnbot/2.0b (+http://search.msn.com/msnbot.htm)

You’ll still see the current version msnbot/1.1 (+http://search.msn.com/msnbot.htm) as Live Search’s primary user agent. We intend to ensure that any robots exclusion protocol you are using is respected. As such, you don’t need to update your Robots.txt file. We plan on crawling at a slow speed during the tests with the updated version.

When the new crawler replaces our current crawler, we will keep you up-to-date here on the Webmaster Center blog. If you have any questions about the Live Search web crawlers, you can post them in our forums.

Additional posts you should read:

Jeremiah Andrick – Program Manager, Live Search Webmaster Center

Join Bing Community
  • this means 2 msn crawler will be visiting my site is it? Was it already rolled out, or will just implement it?

  • The link to the forums is b0rken.

  • can't find papaleonesspaghettihouse.com on live search why ? can on yahoo & google it is my site can you help me thanks DaddyRock@comcast.net

  • Perhaps you haven't submitted your website?

    http://search.msn.com.sg/docs/submit.aspx

    Note, there is always a waiting period for websites to be indexed.

  • I still can't find http://www.questempire.com on live search, i can every where else.

  • If you type in  papa leones spaghetti house  you will not fine my website just reviews of my restaurant. but you can on google or yahoo? Now  put in  papa leones spaghetti house.com  the page will be on top with porn. It does not belong with porn & stuff like that. You should type in    papa leones spaghetti house   press search and the website should appear with all of the reviews.  Why no one knows how to fix this. I have talk to everybody with MSN help thanks YOU CAN CALL 410-456-7278

  • good news,My bolog have been indexed 2 pages.waiting

  • my blog do not get indexed. How i can

  • If you type in papa leones spaghetti house you will not find my website just reviews of my restaurant  But put in papa leones spaghetti house.com the page will be on top.  It does not belong with porn & stuff like that. You should type in papa leones spaghetti house press search and the website should appear with all of the reviews.  Why no one knows I have talk to everybody with MSN help thanks  

    .

    type this in and it should be web page and reviews

    papa leones spaghetti house = just reviews

    type this in it should not be in with porn  and stuff

    papa leones spaghetti house.com= webpage & porn & stuff

    customers  cant find website please help call 410-456-7278

  • how long will it take you to come over here and fix it

  • if Another crawler in my logs show me so ican take it out !

  • You’ll still see the current version msnbot/1.1 (+http://search.msn.com/msnbot.htm) as Live Search’s primary user agent. We intend to ensure that any robots exclusion protocol you are using is respected. As such, you don’t need to update your Robots.txt file. We plan on crawling at a slow speed during the tests with the updated version what does that mean ?

  • Srednarb,

    The new bot is just being tested. You may see it, you may not. We just wanted to make sure that if you saw it, you would know it was a Microsoft bot and not that of someone spoofing us.  

    Thanks

    Jeremiah

  • daddyrock,

    This statement means that we are just testing this update to the bot and we are not rolling it out completely.  We don't want to overcrawl or crawl to hard on your servers so we are limiting the amount of crawling this bot will do.

    Thanks

    Jeremiah

  • Will this crawler actually work unlike the others? Seriously, submitted a site 3 months back, it's only indexed 3/30+ pages despite submitting sitemaps. Google took a couple of days to index the whole lot. No wonder noone uses Live, the content isn't there because you fail to crawl. Clients won't spend money on advertisering on Live if there websites aren't even indexed. I think the Live team need to wake up.

Page 1 of 3 (35 items) 123