How to Index a Magento Site on Google? (Ultimate Guide)

Your site can only show up on Google if it is indexed.

This happens when Google crawler (aka Googlebot) reads your webpage and stores it in its index. 

Google index is like a library of billions of web pages found by Google. The search engine uses the index to show results that match the user query.

If you are looking for a way to index your Magento site on Google, this guide is for you.

Why is Google Indexing Important?

One of the core reasons why indexing matters is to appear on search results. If your Magento webpages are not indexed by Google, they won’t appear in search results.

Primely, indexing a website on Google allows you to get into the race ranking. And with a higher ranking, you can get organic traffic to your page. 

How to Know if a Page is Indexed by Google?

There are two simple ways to check the indexing status of any page:

Inspect the URL in GSC

Open your website’s Google Search Console property and enter the URL in the search bar at the top.

The results will display if the web page is indexed by Google or not.

Use site: Operator 

You can use the site: Google search operator to find a specific URL on Google. This way you can confirm if it’s indexed by Google or not.

Navigate to Google and enter “site:” followed by the URL you want to check.

For example: 

site:https://meetanshi.com/blog/magento-2-seo-guide/

How to Get Google to Index Your Magento 2 Site? 

Here are three ways to get your website indexed by Google: 

• Submit an XML Sitemap
• Manually Request Indexing via Search Console
• Use Google Indexing API

Submit an XML Sitemap

Sitemap is an XML file with all your important URLs. It helps Google crawlers find the pages and index them. You can create a sitemap in your Magento 2 store and submit it to Google for indexing.

Here is how it looks:

Manually Request Indexing via Search Console

Go to your Google Search Console property and enter the URL in the search box. If the URL is not indexed on Google, you can submit an indexing request to make Google index your site. 

Use Google Indexing API

Let’s say you want to submit indexing requests for hundreds (or thousands) of URLs. You can use the indexing API to send URLs to Google for indexing.

Note: Officially, the Google Indexing API is only meant for URLs with News or Job Posting. However, it also works for other types of pages and can be used as an indexing method. 

To use the Indexing API, you’ll need to create a Google Cloud Platform (GCP) service account and add its email address to your Search Console property. Then, you can use the service key file to call the Indexing API and submit indexing requests programmatically in bulk.

For example, here’s the Python code that sends indexing requests for a batch of URLs:

import json
import requests
from google.oauth2 import service_account
from google.auth.transport.requests import AuthorizedSession

# Path to your service account JSON key file
SERVICE_ACCOUNT_FILE = 'path_to_your_service_account_file.json'

# The scope required for the Indexing API
SCOPES = ['https://www.googleapis.com/auth/indexing']

# The URL for the Indexing API
INDEXING_API_ENDPOINT = 'https://indexing.googleapis.com/v3/urlNotifications:publish'

# List of example eCommerce URLs to be indexed
URLS = [
    'https://example-ecommerce.com/product/awesome-shoes',
    'https://example-ecommerce.com/product/stylish-handbag',
    'https://example-ecommerce.com/category/mens-fashion',
    'https://example-ecommerce.com/product/leather-jacket',
    'https://example-ecommerce.com/product/smart-watch',
    'https://example-ecommerce.com/product/summer-dress',
    'https://example-ecommerce.com/product/winter-boots',
    'https://example-ecommerce.com/category/accessories',
    # Add more URLs as needed
]

# Function to get an authenticated session
def get_authenticated_session():
    credentials = service_account.Credentials.from_service_account_file(
        SERVICE_ACCOUNT_FILE, scopes=SCOPES)
    session = AuthorizedSession(credentials)
    return session

# Function to send bulk indexing requests
def send_bulk_indexing_requests(urls):
    session = get_authenticated_session()
    for url in urls:
        # Prepare the request payload
        payload = {
            "url": url,
            "type": "URL_UPDATED"  # Or "URL_DELETED" if you're removing URLs
        }
        headers = {
            'Content-Type': 'application/json'
        }
        
        # Send the request
        response = session.post(INDEXING_API_ENDPOINT, json=payload, headers=headers)
        
        # Handle the response
        if response.status_code == 200:
            print(f'Successfully indexed: {url}')
        else:
            print(f'Failed to index {url}. Response: {response.content}')

# Call the function to send bulk indexing requests
if __name__ == "__main__":
    send_bulk_indexing_requests(URLS)

Note: Submitting an indexing request to Google via API does not guarantee indexing.

Magento 2 Extension to Automatically Send Google Indexing Requests

You can use Magento 2 Google Indexing API extension to automatically submit new or updated pages. This is the easiest and fastest way to get your pages crawled and indexed by Google (without writing a single line of code).

The extension integrates Indexing API with Magento 2 and allows you to:

• Send indexing requests for new content
• Bulk submit a list of URLs for indexing
• Auto-send indexing requests on content update

Frequently Asked Questions

Why is Google Not Indexing All Products in Magento 2?

If your products are not indexed, check the robots.txt file and the tag associated with it. If your product page has a noindex tag, then Google is not allowed to index it. You will need to change the tag setting. 

How to Fix “Crawled – Currently Not Indexed” GSC Issue?

Google may crawl a web page, but not index it because of many reasons like poor page experience, thin content, or presence of spammy links.

How to Hide Pages from Google in Magento 2?

A simple way is to add a noindex meta tag in the <head> of the page you want to hide. 

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "FAQPage",
  "mainEntity": [{
    "@type": "Question",
    "name": "Why is Google Not Indexing All Products in Magento 2?",
    "acceptedAnswer": {
      "@type": "Answer",
      "text": "If your products are not indexed, check the robots.txt file and the tag associated with it. If your product page has a noindex tag, then Google is not allowed to index it. You will need to change the tag setting."
    }
  },{
    "@type": "Question",
    "name": "How to Fix “Crawled - Currently Not Indexed” GSC Issue?",
    "acceptedAnswer": {
      "@type": "Answer",
      "text": "Google may crawl a web page, but not index it because of many reasons like poor page experience, thin content, or presence of spammy links."
    }
  },{
    "@type": "Question",
    "name": "How to Hide Pages from Google in Magento 2?",
    "acceptedAnswer": {
      "@type": "Answer",
      "text": "A simple way is to add a noindex meta tag in the <head> of the page you want to hide."
    }
  }]
}
</script>

Maximize SEO with Google Indexing

Indexing is a must to stand a chance of ranking on Google. However, it doesn’t mean your site will start ranking instantly.  Once your site is indexed, you can focus on optimizing it for SEO to climb higher in the rankings and get search traffic.