Update on MSN/Live and Yahoo Traffic

It's been almost a month since I started working on getting more traffic from Yahoo and MSN/Live.com search engines on one of my sites, but my efforts seem to be paying off. All the changes were on site, mainly changes to my robots.txt, sitemap.xml and pinging updates when the site ads new content.

Compare to the previous period, traffic improved across all major search engines. Google was already giving me a good amount of visitors but traffic from Google increased over 17%. Traffic from Yahoo! search increased 50% and MSN/Live.com traffic had the biggest improvement of over 180%. Ask.com traffic improved by over 90%.

Google still is my major source of search engine traffic by far but these results seem promising.

Live.com has indexed a lot more of this site as a result of the changes. Before the changes, Live.com had only indexed about 1% of the site, now it's over 10% of the site and MSNBot seems to be visiting more frequently. Every time I check the number increases and my domain score is 5 out of 5.

The increased index pages helped bring some more long tail keyword traffic from Microsoft's search engines. Since there are still 10 times as many pages that can be indexed by MSNBot that haven't, I'm hoping traffic from MSN and Live.com continue to grow. Yahoo seems to have indexed the pages well. While I'm expecting search traffic growth from Yahoo!, my expectations are less.

One thing to note about traffic from Live.com is that log analyzers such as AWStats may be misleading. Microsoft has a crawler they use to detect cloaking. It acts like a normal website visitor and downloads all images, stylesheets, javascript, and so on. The problem is that when it accesses a page, it uses a referrer header to make it appear is if it came from a search on live.com. While Microsoft claims to have fixed the problem, I still see this behavior across different sites and the invalid search engine keywords being affected.

Google Anlytics seems to recognize these fake search engine referrers and doesn't report them as genuine search engine traffic.