Yesterday marked a big day for the SEO industry. Previously, we had limited understanding of how Google AI Overviews (AIOs) were appearing for individual websites, a feature that is quickly becoming more prominent within search results.
Semrush has now stepped up to the challenge by making AIO tracking far more accessible within their toolset, now showing the filtering beyond the Position Tracking confinement (how other SEO tools presented their data), to now being available within Organic Research.
This change now allows SEOs to far more easily perform analysis on AIO rankings at scale. I’ve been playing around with the new AIO filter within Organic Research for large-scale sites that I work with and have some learnings to share – here’s what I’ve learnt so far.
How Tracking Works
When it comes to AI Overview tracking within Google Search Console, this should be considered the most valuable dataset for AIO performance, being 1st party data. The problem here is that there is no filtering available for AIOs, which leads to a lot of questions related to performance and impact.
In comes a tool like Semrush. While imperfect at projecting organic traffic and keyword volumes, the tool does a good job considering it is based on estimates. By using the data within Semrush and our understanding of how AIO data is recorded in GSC, we can at least piece together a reasonably accurate picture for AIO performance.
Mobile vs. Desktop
As a first step in understanding how AIO tracking works in Semrush, it is important to understand the difference among how AIOs appear on mobile and desktop. With the new AIO format on desktop, links to publishers are shown prominently for the AIO result, much like how a Featured Snippet would represent reference links.
On mobile, the situation is currently much different. This is an important aspect to get your head around, where the citation links themselves are only shown after interaction. Semrush has the ability to filter by mobile within the Organic Research feature. When filtering by mobile with ‘SERP Features on SERP’ for AI Overviews, you will see results. But when using ‘Domain ranks’, you will not see any results, which is different to desktop. With this key difference in mind, sites will likely be getting less traffic from mobile with AI Overviews compared to desktop, but the CTR will be higher – with the recording method being similar to how Merchant Listings operate.
Link Reporting
When it comes to the Organic Research reporting for AIOs in Semrush, the placements that are recorded on desktop are for the pages that are visible on page load (without interaction). This is how I believe external tracking should work for this feature, at least at the most basic level.
Based on my testing, Google is currently showing 2-3 references from sites within AIOs on desktop, with the 3rd result often being more hidden (or completely hidden) when compared to the first two. If placed within either of these results, Semrush will record that as a single ranking within their dashboard.
Comparison to Featured Snippets
While this wasn’t always the case, I believe that Google has now done a good job of rolling out AI Overviews more broadly in a way that doesn’t impact publishers negatively. This is in comparison to other alternatives out there, which send very few clicks to publishers by design. When comparing the current desktop AIO format to the long-standing Featured Snippets feature (which still exists), the format isn’t all that different.
Sources for AIOs are presented on the right of the result that has been generated, with the ability to scroll through additional sources and different components of the answer if necessary. Now, this doesn’t mean AIOs are necessarily a better answer to Featured Snippets in most cases I’ve seen, but we’ll let the users decide that one for themselves.
An interesting component of AIOs is that being visible within the AIO allows the exact URL that’s featured to also rank directly below. This is something that is not possible with Featured Snippets since deduplication years back. There are cases where Google still displays Featured Snippets below AIOs, where sites can appear with the same URL in both. But if featured twice within each of these sections, you won’t find that same URL appearing directly below within the standard organic listings.
Eligible Countries for Analysis
Previously, the best country-focused dataset we had for AIOs was within Sensor. The tool only reports on a limited set of countries, with the Organic Research tool having access to more country data. Based on what I can see within Semrush, there are now datasets for the US, the UK and India that have data being recorded within the dashboard for sites, allowing for AIO analysis within only these regions at this time.
Key Reporting Comparisons
Now that we have access to AIO tracking within Organic Research in Semrush, this opens up many avenues for analysis at scale for large domains. As an example of some of the reports that you can create for clients, I’ll use Wikipedia as an example to show the AIO impact.
AIOs on SERP vs. Ranking AIOs
For Wikipedia, they rank for 156.9M keywords within the first 100 results of Google according to the Semrush index. Out of the 156.9M keywords, 151.9K have AI Overviews as a SERP feature on desktop. This is 0.1% of all Wikipedia keywords that they rank for. Out of the 151.9K keywords that have an AIO as a SERP feature, I can also see that Wikipedia ranks within the visible citations on load for 17.4K of those, occupying 11.45% of the possible placements.
Top 10 Ranking Without AIO Placement
There is some research behind results that rank on the first page of Google having a far greater chance of appearing within a Featured Snippet. Based on my testing, it looks like AIOs operate in a similar way to this for the top visible citations at least. Within Semrush, you’re able to select positions in the top 10 results where the domain doesn’t rank within the AIO visible citations. For Wikipedia, I can see that there are 33K keywords where this is the case.
Competitor AIO Comparison
While Wikipedia doesn’t exactly have direct competitors, there are ways for sites to do competitor comparisons for AIOs using Semrush. As an example, Reddit overlaps with Wikipedia within Google’s search results across around 50.3M keywords. While there isn’t currently the ability to do SERP feature filtering within the Keyword Gap tool, I can however see how Reddit is performing within AIOs compared to Wikipedia.
Reddit ranks for 205.2M keywords with 470.3K of those having AIOs on them, making up 0.23% of all keywords. This is higher than the 0.1% for Wikipedia but is still very low. Out of the 470.3K keywords that have AIOs on them, Reddit currently ranks within only 35 of them. This is extremely low compared to the 17.4K for Wikipedia based on the size of each site, clearly pointing to a more considered approach by Google to not allow UGC to power too much of the AIO function within Search.
Tracking Caveats
Logged-in vs. not logged in
It was only recently that data for users that aren’t logged in to Google was able to be recorded, as AIOs weren’t showing for this experience yet, which has since changed. It is important to note that I do still sometimes see discrepancies among what a logged in account shows compared to incognito, but it is so massive of a difference that we can disregard all of the non-logged in data.
Non-triggered AIOs
I’ve noticed that within a small portion of the Semrush data that there appear to be AIOs that are generated within tracked search results that haven’t been recorded. I’m not sure if this is a delay in how the tracking works for Semrush, but it does mean that there is the potential that the data is larger than what is being reported. Either this, or this could be related to AIOs that start generating but don’t end up showing (which I still see sometimes).
General testing volatility
AIOs seem to have found some form of stability more recently, but it’s not like they are a stagnant SERP feature. Just like Featured Snippets, Google is constantly testing out which sites have the best answers to queries, resulting in the potential of many shifts over the course of a single day. Generally speaking, I tend to find that there is a site or two that is frequently referenced within the AIO for a lot of queries, so there is still value in reporting on this data.
Note: I have also noticed that for some queries, mobile data is showing within Semrush for some very large sites for the ‘Domain ranks’ filter for AIOs. Based on what I can see, this looks to be more of an inconsistency in how results are tracked by device type, which I would expect Semrush will be looking to refine over time.
Ranking not assigned to position
Something that was raised by Suganthan on Twitter is that the AI Overview icon within Semrush doesn’t currently have a ranking assigned, even though the ranking is essentially #1. I believe this is another aspect that Semrush will look to address over time so the feature is tracked more similarly to a Featured Snippet (which shows as #1).
Final Thoughts
Overall, I’m excited by the ability to track Google AI Overviews more effectively at scale for my clients. The fact that they wouldn’t show for non-logged in users made this tracking experience quite fractured, but we’re now moving in the right direction. The new Semrush dataset helps us understand exactly how prominent AI Overviews are for sites and allows for understanding an additional piece of the puzzle that was previously missing.
