To stop undesirable information within the look for indexes, webmasters can instruct spiders to not crawl selected files or directories throughout the regular robots.txt file in the root directory of your area. Moreover, a page might be explicitly excluded from the internet search engine's databases by using a meta tag https://news.rafeeg.ae