For successful indexing of the page by search bots, the HTTP response code of the server must be 200.

Additional Information:

• [Checking the server response of the internal pages of the site](/tools/http-response/
• List of status codes

Description:

A robots.txt file is a list of restrictions for search robots or bots that visit a site and crawl information there. Before crawling and indexing your site, all robots go to the robots.txt file and look for rules.

The robots.txt file is located in the root catalogue of the site. It must be available on the URL: pr-cy.io/robots.txt

There are several reasons to use a robots.txt file on your site:

1. to remove duplicate content;
2. to hide unwanted information;
3. to limit the indexing speed.

Description:

A Sitemap file is a file with information about the pages on the site to be indexed. With this file you can:

• tell search engines which pages of your site need to be indexed;
• find out how often the information on the pages is updated;
• understand which pages are most important to index.

Additional Information:

• Official Documentation

For the promotion of commercial sites, the confidentiality of the exchange of information between the server and visitors is important. This increases the loyalty of potential customers to the resource, increases the level of trust, affects the conversion and growth of positions in the search results for almost all requests.

Description:

Technically, domains with www and without www are two different resources; search engines index and rank them separately, and links will have different weights. This can lead to:

• Decrease in search results
• Filter, because a search engine may mistake one site for a duplicate of another;
• Problems with authorization on the site and other functionality that uses cookies;

The problem is solved by a 301 redirect and an indication of the main mirror to the search engines. From the website promotion point of view, a domain without www is better, because it is not a third-level domain, and its length will always be shorter.

Download speed directly affects behavioral factors: the faster the download, the fewer bounces. The Google robot visits slow sites less often. This affects the effectiveness of promotion, such sites are rarely indexed. Loading speed is one of the main factors in Google ranking.

Description:

While requesting a non-existent page, the server should return a 404 error which is"page not found."

If the server is set up incorrectly and a 200 error returns, then the page exists. In this case, search engines may index all pages of the site with errors.

Set up the site in such a way that while requesting non-existent pages a 404 response code "page not found" or a response code 410 "page deleted" appear.

Description:

During requesting a non-existent page, the server displays a standard page with a 404 error. For the users’ convenience we recommend making a unique 404 page and adding a backlink to the site on it.

Don't forget to prolong your domain name. It’s better to involve auto-renewal with your registrar. After the end of the domain registration, there is a chance to lose access to the domain.

IP addresses ever found on the site. And also the sites that have the same IP address.