How to allow all websites crawlers using Robots.txt file

Step 1: Login to the Cpanel.
Step 2: Open File Manager and go to the root directory of your website.
Step 3: The Robots.txt file should be in the same location as the index file of your website.
Step 4: Right click on the phpinfo.php file and choose code editor to edit

User-agent: *
Allow: /

Step 5: Add the lines of the code provided above to the file and click on ‘Save’ to save the changes made.

Click here to view this article as Oryon knowledge base video tutorial playlist.

Related Articles