When working on a site for a customer under a development domain I noticed that the ‘Last 5 Search Terms’ and the ‘Top 5 Search Terms’ entry on the dashboard had thousands of hits.
That was impossible because the site wasn’t in production and I had only done a few catalog searches myself. So after a little research, it turned out that these large numbers were generated because the Google bot had been indexing the ‘Search Terms’ page in the footer of the website. The Search Terms page displays a tag cloud of the most popular search terms. Following a link in the tag cloud is the equivalent of issuing an internal store search. The Google bot and other search engine crawlers have been following links in the tag cloud repeatedly (maybe once for every page in the website) and so the popular search term numbers went up.
This is a problem, because it taints the internal search metrics for the store which are really important for analytics and marketing.
The easiest fix is to tell the search bots not to index the Search Terms page. Add the following line to the robots.txt file found in the root of your magento app.
If you don’t have a robots.txt file there, create one and add the following lines. Note: Normally you would want a bunch of other entries in robots.txt, to make the indexing of your store more efficient and to avoid duplicate content, but that’s another post waiting to be written.
User-agent: * Disallow: /catalogsearch/term/ Sitemap: http://www.domain.co.uk/sitemap.xml
To reset the search term count go to Catalog > Search and delete all the entries.