Bing

Another crawler in your logs

December 11, 2008, 06:05 PM by Webmaster Center team | 33 Comments

Live Search is committed to continuing to improve how we discover and crawl your content. In the coming weeks, we will be testing an update to MSNBot, which may show up as a new crawler name in your referrer logs. The new crawler user agent string will appear as:

msnbot/2.0b (+http://search.msn.com/msnbot.htm)

You’ll still see the current version msnbot/1.1 (+http://search.msn.com/msnbot.htm) as Live Search’s primary user agent. We intend to ensure that any robots exclusion protocol you are using is respected. As such, you don’t need to update your Robots.txt file. We plan on crawling at a slow speed during the tests with the updated version.

When the new crawler replaces our current crawler, we will keep you up-to-date here on the Webmaster Center blog. If you have any questions about the Live Search web crawlers, you can post them in our forums.

Additional posts you should read:

Jeremiah Andrick – Program Manager, Live Search Webmaster Center

subscribe

Comments

Srednarb

Posted On December 12, 2008, 02:54 AM

this means 2 msn crawler will be visiting my site is it? Was it already rolled out, or will just implement it?


Ian M

Posted On December 15, 2008, 09:40 AM

The link to the forums is b0rken.


gene

Posted On December 16, 2008, 05:57 PM

can't find papaleonesspaghettihouse.com on live search why ? can on yahoo & google it is my site can you help me thanks DaddyRock@comcast.net


Shirley

Posted On December 18, 2008, 07:44 AM

Perhaps you haven't submitted your website?

http://search.msn.com.sg/docs/submit.aspx

Note, there is always a waiting period for websites to be indexed.


Quest Empire

Posted On December 19, 2008, 10:00 PM

I still can't find http://www.questempire.com on live search, i can every where else.


gene

Posted On December 20, 2008, 12:12 AM

If you type in  papa leones spaghetti house  you will not fine my website just reviews of my restaurant. but you can on google or yahoo? Now  put in  papa leones spaghetti house.com  the page will be on top with porn. It does not belong with porn & stuff like that. You should type in    papa leones spaghetti house   press search and the website should appear with all of the reviews.  Why no one knows how to fix this. I have talk to everybody with MSN help thanks YOU CAN CALL 410-456-7278


dangdang

Posted On December 20, 2008, 12:23 PM

good news,My bolog have been indexed 2 pages.waiting


sami

Posted On December 24, 2008, 12:42 AM

my blog do not get indexed. How i can


gene

Posted On December 29, 2008, 09:13 PM

If you type in papa leones spaghetti house you will not find my website just reviews of my restaurant  But put in papa leones spaghetti house.com the page will be on top.  It does not belong with porn & stuff like that. You should type in papa leones spaghetti house press search and the website should appear with all of the reviews.  Why no one knows I have talk to everybody with MSN help thanks  

.

type this in and it should be web page and reviews

papa leones spaghetti house = just reviews

type this in it should not be in with porn  and stuff

papa leones spaghetti house.com= webpage & porn & stuff

customers  cant find website please help call 410-456-7278


Anonymous

Posted On January 03, 2009, 01:54 PM

how long will it take you to come over here and fix it


Anonymous

Posted On January 03, 2009, 02:49 PM

if Another crawler in my logs show me so ican take it out !


Anonymous

Posted On January 03, 2009, 02:54 PM

You’ll still see the current version msnbot/1.1 (+http://search.msn.com/msnbot.htm) as Live Search’s primary user agent. We intend to ensure that any robots exclusion protocol you are using is respected. As such, you don’t need to update your Robots.txt file. We plan on crawling at a slow speed during the tests with the updated version what does that mean ?


Webmaster Center team

Posted On January 04, 2009, 07:40 PM

Srednarb,

The new bot is just being tested. You may see it, you may not. We just wanted to make sure that if you saw it, you would know it was a Microsoft bot and not that of someone spoofing us.  

Thanks

Jeremiah


Webmaster Center team

Posted On January 04, 2009, 07:46 PM

daddyrock,

This statement means that we are just testing this update to the bot and we are not rolling it out completely.  We don't want to overcrawl or crawl to hard on your servers so we are limiting the amount of crawling this bot will do.

Thanks

Jeremiah


annoyed developer

Posted On January 05, 2009, 07:50 AM

Will this crawler actually work unlike the others? Seriously, submitted a site 3 months back, it's only indexed 3/30+ pages despite submitting sitemaps. Google took a couple of days to index the whole lot. No wonder noone uses Live, the content isn't there because you fail to crawl. Clients won't spend money on advertisering on Live if there websites aren't even indexed. I think the Live team need to wake up.


Write a Comment

This information is published in the Community – it’s public.
 

Save Comment Remember Me?

Welcome

to the Microsoft Bing community

Remember, don't post your personal information!