Robots.txt Is often a file that exists on the root directory of each Site and can be employed to instruct engines like google on which directories/files of the web site they can crawl and incorporate in their index. Whether or not you work on Web optimization yourself, delegate it to https://news.rafeeg.ae