New "Bingbot" Will Crawl Non-optimized Sites More Easily

[alabuba]

Microsoft has announced that it will be bringing the Bing web crawler out of beta on October 1st. It will be rebranded as "the Bingbot" and replace the existing msnbot. "It will still honor robots.txt directives written for msnbot, so no change is required to robots.txt file(s)," a Bing representative tells WebProNews.

"Improvements to the bot enable more efficient crawling, and increase the ability to crawl content on sites not optimized for search," he says.
 
Rick DeJarnette has more about the change on the Bing Webmaster Blog:

Instead of the old msnbot 2.0b showing up in your server logs, the updated user agent will be:

Mozilla/5.0 (compatible; bingbot/2.0 +http://www.bing.com/bingbot.htm)

The HTTP header From field will also change as shown below:

From: msnbot(at)microsoft.com

will become

From: bingbot(at)microsoft.com

If Bing finds separate sets of directives for Bingbot and for other crawlers, directives for bingbot will take precedence, the company says.

I find the part about increasing the ability to crawl content on sites not optimized for search to be particularly interesting. I wouldn't exactly call this an invitation to ignore SEO. Obviously Google is still the biggest search engine anyway, but even as far as Bing is concerned, good SEO practices will likely still help your rankings.

Also keep in mind that optimizing for Bing is becoming increasingly important. Not only is Facebook giving more reason for people to search (where Bing provides the web results), but the Yahoo/Bing integration will be here (likely) before the holidays.

Posted on Mon, 28 Jun 2010 18:05:05 +0000 at http://feedproxy.google.com/~r/SEO-News-...ore-easily
Comments: http://www.webpronews.com/topnews/2010/0...y#comments