Bing blogs

This is a place devoted to giving you deeper insight
into the news, trends, people and technology behind Bing.

msnbot and non-existant files

Webmaster

Webmaster
This group is devoted to Bing Webmaster Tools discussions.

msnbot and non-existant files

This question has suggested answer(s)

for many many years, msnbot has been crawling my sites looking for files that have never existed... i'm trying to figure out why...

the filenames have changed slightly in recent times but they have been similar in structure since the beginning... they are something like 000092601_00002.temp0001.htm... in other words, 9 numbers underscore 5 numbers dot temp 4 numbers dot htm... the search for these is all over my server's directory tree...

i'll emphasize once more that these files have never existed on my site and i have no clue how msnbot may have picked them up...

now, how can i get msnbot to stop polluting my logs looking for them???

All Replies
  • Sorry to tell you there isnt a way to stop msnbot from doing what it's doing.

    I am having the same issue and it appears you can't tell a scripted computer program what to do.

  • Can you show us the URL of your site?

    Also, I think robots.txt file can help in this regard. But before that, do look in your site source code: to see if there are some links, generated by mistake, that points to the URLs you mentioned.

  • Hi

    Could you let me know your URL?

    Many Thanks

    Archie

  • no, i can't/won't show the URL unless it is specific to a msnbot operator...

     

    as for "source code", the entire site where these cr4p URLs are looking is pure handcrafted html... there is no, repeat, absolutely NO scripting of any kind... it is and always has been pure handcrafted HTML...

  • Archie

    Hi

    Could you let me know your URL?

    Many Thanks

    Archie

    why??? are you a msnbot operator??

  • wkitty42

    Archie

    Hi

    Could you let me know your URL?

    Many Thanks

    Archie

    why??? are you a msnbot operator??

    No, but if you aren't willing to share your URL then there is no point in asking for support in a public forum.  The only reason people want your URL is to have a look at the site to see what might be causing the problem you are having, and if you're not willing to provide it there's not much that can be done on the forum.  You should contact support for help with this issue:

    https://support.discoverbing.com/eform.aspx?productKey=bing&ct=eformts&scrx=1

  • ummm... somehow the words "support forum" and the rest of your reply don't fit... i am here, trying to contact support about this years old problem... if this is not where support hangs out, then bing has more to fix in their links and pages because this is where they directed me for msnbot support...

    now, as for "the errors i'm having"... I'M not having any errors... the msnbot is having the errors by looking for files that have NEVER existed on my site... please understand this because i cannot use any easier words in english to state it... msnbot has been looking for these goofy files for years and they have never existed nor have they ever had any way of existing...

    pure handcrafted HTML code does not generate temporary files for any reason, right??

    thanks for that link... i'll give it a try and see what comes of it... why isn't it listed as a webmaster support link in the rest of bing? it should be if this forum is not the proper place for webmasters to contact bing operators directly to solve problems like this...

    here's another question for everyone to ponder... how many years, yes years, does it take of getting a 404 response before something is finally removed from the index and the searches? seriously... if msnbot gets a 404 for 3 months in a row, it stands to reason that the URL no longer exists and should be dumped from the search and the bot... who's the mental giant that came up with the idea that "404 Not Found" means that it just might be found if we keep looking several thousand more times :?

  • Archie

    sadly this link takes me right back to help.live.com which is where i started this convoluted road to hell trying to get this problem fixed :(

     

  • Archie

    wkitty42

    Archie

    Hi

    Could you let me know your URL?

    Many Thanks

    Archie

    why??? are you a msnbot operator??

    No, but if you aren't willing to share your URL then there is no point in asking for support in a public forum.

    then again, is there something wrong with my profile?

  • Brett Yount is the Bing Webmaster Tools Program Manager and he reads most of the posts here. He's helpful to site owners who are experiencing issues. Try to contact him so that he can help you out.

    Maybe you can go to his profile page and drop a message.

  • Quality Directory
    Brett Yount is the Bing Webmaster Tools Program Manager and he reads most of the posts here. He's helpful to site owners who are experiencing issues. Try to contact him so that he can help you out.

    Maybe you can go to his profile page and drop a message.

    yes, now that i can see a bit more about who is who, i might do that... i see that you and yasirwazir are also pretty active and yasirwazir was one of the ones who requested my site URL (which is listed in my profile)...

  • yasirwazir
    Can you show us the URL of your site?

    ummm... see my profile ;)

    yasirwazir
    Also, I think robots.txt file can help in this regard. But before that, do look in your site source code: to see if there are some links, generated by mistake, that points to the URLs you mentioned.

    if robots.txt can work with regex links, then it might help but that's like pulling up all your sod to keep the cow from eating it...

    as written before, in first message, the site is all pure HTML code and there never have been any links to anything like the example name i posted in my first message... why would i do something as st00pid as that? if i want to send someone to foobar.html than it is linked as that and not some 000014325_00001.temp0001.htm name that doesn't makes any sense to start with... understand?

  • It seems to me that the answer lies with dynip.com.  Even though your code is straight html, you are a sub domain of a much larger domain that you don't control. Perhaps you could ask them. 

    Search engines just don't make up urls to crawl.  If you read through the forums you will see that people have enough issues getting their real content crawled, much less non-existent content.

     

  • wkitty42

    Archie

    sadly this link takes me right back to help.live.com which is where i started this convoluted road to hell trying to get this problem fixed :(

     

    In that case do it this way:

    1 - Go to http://www.bing.com/

    2 - Click "Help" - bottom right hand corner

    3 - Click "Get More Help" - again bottom right hand corner

    4 - And then click "get Support" - this will give you a contact form

  • wkitty42

    if this is not where support hangs out, then bing has more to fix in their links and pages because this is where they directed me for msnbot support...

    As far as I can tell there is only one Bing representative who actually posts on these forums so a reply from Bing is not guaranteed if you ask for support here