To prevent undesirable material within the search indexes, webmasters can instruct spiders not to crawl certain data files or directories in the conventional robots.txt file in the foundation Listing from the area. Also, a site is often explicitly excluded from the internet search engine's databases by utilizing a meta tag https://www.facebook.com/acourseinmiraclesdavid/