Why is Getting Indexed by Search Engines So Hard?
Getting indexed by search engines can be difficult. Here’s what to do about it.
All public-facing websites rely on Google to some degree. As an affiliate marketer, getting indexed is vital. Indexing, by Google especially, makes it possible for potential customers to find you. Unfortunately, many pages never get indexed by search engines. Some pages will go weeks before getting recognized by Google, if ever.
Crawled – currently not indexed
Google Search Console reports several statuses for unindexed pages. The “crawled-currently not indexed” status means that Google visited your site and chose not to index it. This is typically a content quality issue. If one of your pages has this status, add unique value to the page:
- Use unique meta titles, descriptions and copy on all pages.
- Avoid copying content from other sources
- Use canonical tags to consolidate duplicate content
- If you have lower quality sections of your site, use the txt file or the no-index tag to prevent Google from crawling those pages.
If your pages receive this status from Google Search Console, do some research on Google to see which pages are ranking for your topic or keywords. See what they’re writing about, how much unique text they have on the page, what they are linking to, etc. Update your content to be on par with your SERP (search engine result page) competitors. The highest quality content expands on a topic and provides unique information.
Discovered – currently not indexed
This status covers a wide range of issues from insufficient content quality to crawling issues. Many large websites experience this issue. You may receive this indexing status if there is a:
Quality issue: Google thinks your pages aren’t worth crawling. If this happens, Google can look for a pattern in the URLs of those pages that are low quality to avoid indexing additional pages on your site.
Crawl budget issue: There were too many URLs in the crawling queue, and these pages may be crawled and indexed at a later time.
If you receive this status, there are some things you can do:
- Check to see if there is a pattern of pages that fall into this category. Then look for similarities in these pages. Are there no links pointing to these pages? Are there fewer words on these pages? Is the content lower quality?
- Add low quality pages to your robots.txt file or no-index them, so Google doesn’t waste time attempting to crawl them and can move onto your higher quality pages.
Duplicate content is a common issue. Duplicate content is content that appears in more than one place on the internet. If the same content appears on more than one unique website address you have duplicate content. It can be hard for search engines like Google to decide which version of the content is more relevant to show in SERP results. Duplicate content can be caused by numerous reasons including:
- If you have several versions of the same page created for different countries, these pages can end up unindexed.
- If you are an affiliate using the exact content an advertiser or network provides, your competitors or other affiliates could be using this content as well.
To avoid duplicate content, provide a unique take on the products or services you are providing.
How can I check my website’s index coverage?
Getting your pages indexed by search engines is important. It’s how your customers or target audience can find you. It’s important to check your website coverage regularly to catch any issues.
The best place to check your website’s index status is in the Index Coverage Report in Google Search Console. Look at the number of excluded pages, and try to find a pattern among them. Getting indexed by search engines doesn’t have to be impossible. If you see one of the statuses above, take the recommended actions to fix them. Once the issues are addressed, resubmit your sitemap for indexing.