Since the release of Google’s AI Overviews (AIOs), a concerning amount of AIO responses have been showing up for mental health searches. People who search mental health topics are often facing stressful, confusing situations, making that search moment a vulnerable one, and one where they are trusting the information they receive will be reliable.
As Google might say, these are without a doubt Your Money/Your Life (YMYL) searches where information gathered from search results may have a major impact on your life. Google has said they set higher standards on the content and websites that show up for YMYL searches. Having an AIO present people with information that could significantly impact their lives alarmed us.
As a mental health content site, ChoosingTherapy.com believes that online mental health information should be written by people with relevant expertise so that the content is as accurate, helpful, and trustworthy as possible. This is precisely why all the writers at ChoosingTherapy.com are licensed therapists and all content is reviewed by a medical professional and held to extremely high editorial standards. It is also why all of the therapists writing for us sign a No AI Pledge.
Our concerns about AI in mental health, and specifically the prevalence of AIOs answering mental health queries inspired us to investigate exactly how AI overviews are affecting the mental health space.
On June 6th, 2024 we used Ziptie.dev, Ahrefs, and Semrush to analyze a total of 3,720 mental health keywords covering topics on mental health disorders (ie “anxiety”), therapy techniques (ie “cognitive behavioral therapy”), and medication queries (ie “how to get prescribed Adderall”).
We were mainly interested in answering the following questions:
- How often do AI overviews show up for mental health searches?
- What websites are being cited as sources in AI overviews?
- Is the information presented in AIOs potentially harmful?
While answering these questions, we uncovered other interesting insights as well. Here’s just some of what we’ve discovered so far:
Key Takeaways
Over 38% of mental health keywords resulted in an AI overview
Websites with higher domain authority are more likely to be an AIO source
Sources cited in the AIOs usually also rank in the top 10 for the same keywords
The top 5 sites referenced in AIOs are NIH.gov, Healthline.com, ChoosingTherapy.com, MedicalNewsToday.com, and WebMD.com
We didn’t find examples of obviously harmful information in the AI overviews (yet)
How Often Do AI Overviews Show Up for Mental Health Searches?
We began analyzing 3,720 mental health keywords on June 6th, 2024, and discovered that 26% of queries resulted in an AI overview (978 out of 3,720 queries).
However, after some analysis of the keywords, we noticed almost a fourth were names of specific medications and companies. The keywords that contained the name of a medication had very few AI overviews (6.8%). The keywords that contained a company name triggered no AI overviews at all (0%).
We believe it’s likely that Google’s understanding of searches that include the names of medications and company names may be deemed “transactional searches.” Users may find an AIO less helpful for transactional searches.
When we filtered out the transactional searches to isolate the “informational searches” we discovered that 38.32% of mental health keywords resulted in an AI overview (896 out of 2,338 queries).
This is significantly higher than the 16% found for ecommerce queries, the 15% found for a wide variety of queries by SEO clarity, and the 8.71% for general queries reported by SEranking.
Interestingly, it’s very close to the 42% found for the general health vertical, which begs the question, why on earth is Google showing significantly more AI overviews for YMYL searches?
Does Domain Authority Matter for Sources in AI Overviews?
We wanted to see if a website’s authority affected its chances of being cited as a source in AI overviews for mental health keywords. We started by looking at Domain Rating (DR) as measured by Ahrefs.
Of the 3,720 mental health keywords we checked, 978 have an AI overview. When looking at the 978 AI overviews, high DR websites had a significantly higher chance of being listed as a source. We refer to a website’s chance of being presented as a source in an AIO as “AIO source frequency.”
As the chart above makes clear, websites with a DR greater than 80 are more likely to be cited in an AIO than a website with a DR lower than 80. Importantly, websites with a DR greater than 90 are far more likely to be cited than websites with a DR less than 90.
Apart from one outlier, websites with a DR lower than 80 have an AIO source frequency of less than 5%. Further, with only two exceptions, sites with a DR lower than 90 have an AIO source frequency of less than 10%.
Below is a table documenting the websites with the highest AIO source frequency for mental health queries (based on the 3,720 keywords we analyzed):
| Website | Domain Rating (DR) | AIO Source Frequency |
|---|---|---|
| nih.gov | 94 | 41.7% |
| healthline.com | 92 | 30.9% |
| medicalnewstoday.com | 91 | 27.1% |
| choosingtherapy.com | 75 | 26.5% |
| webmd.com | 92 | 26.2% |
| verywellmind.com | 90 | 22.5% |
| clevelandclinic.org | 91 | 18.9% |
| mayoclinic.org | 92 | 17.7% |
| psychcentral.com | 88 | 14.0% |
| www.nhs.uk | 92 | 13.9% |
| youtube.com | 99 | 13.1% |
| psychologytoday.com | 92 | 10.5% |
| verywellhealth.com | 89 | 8.8% |
| mind.org.uk | 89 | 6.3% |
| helpguide.org | 87 | 6.2% |
| talkspace.com | 80 | 6.0% |
| psychiatry.org | 87 | 6.0% |
| goodrx.com | 83 | 5.1% |
| hopkinsmedicine.org | 90 | 4.8% |
| wikipedia.org | 91 | 4.5% |
| betterhelp.com | 82 | 4.5% |
Out of the top twenty sites for AIO source frequency, the one outlier is our own website, ChoosingTherapy.com, with a DR of 75.
Why might ChoosingTherapy.com be outperforming in this study? Perhaps it is because ChoosingTherapy.com is a highly trusted entity whose content is written by licensed therapists and medically reviewed. However, since we made an effort to study a broad set of keywords that were representative of the mental health field at large, it remains possible that there was bias in the keywords we selected for this study.
Does the Type of Website Matter?
Next, we looked at whether the type of website impacted the likelihood of being cited in an AI overview. Namely, are citations dominated by government sites, major non-profits, elite universities, or research hospitals? Do well-known content websites like WebMD get cited? What about less well-known sites, like a local therapist’s small blog?
We discovered that:
- Certain government websites excel at capturing citations
- 72% of the total citations belong to the top 20 cited websites
- Well-known, established brands seem to get cited far more frequently
Here’s the breakdown of the data and analysis:
We analyzed the 116 sites that had at least 5 citations. Within the 978 search results with AI overviews, there were a grand total of 4,812 citations for these sites. We then looked at how many of these citations the various types of websites were able to capture.
The first thing we looked at was the effect of a website’s top-level domain (TLD), such as .com, .org, etc. on AIO citations as outlined in the graph below.
While the percentage of citations captured by dot govs appears much lower than dot coms and dot orgs, they actually captured a disproportionate amount of citations.
Of the 116 websites we analyzed, dot govs only made up 5.2% of all sites and yet captured 14.5% of citations. On the other hand, dot coms made up 53.5% and captured 58.4% while dot orgs only captured 18.3% of citations despite making up 27.6% of total sites.
| Top-level Domain (TLD) | Percent of Total AIO Citations | Percent of Total Sites |
|---|---|---|
| gov | 14.5% | 5.2% |
| com | 58.4% | 53.5% |
| org | 18.3% | 27.6% |
| uk | 3.5% | 1.3% |
| org.uk | 1.7% | 2.6% |
| edu | 0.9% | 1.7% |
| int | 0.5% | 0.9% |
| gov.au | 1.3% | 1.7% |
| other (.net, .ca., .io, etc.) | 0.9% | 5.2% |
While this is interesting, it doesn’t quite tell the full story. The majority of citations captured by dot govs came from just one or two websites. Also, TLD can be a poor indicator of a website’s actual true nature.
In light of this, we decided to do a more detailed analysis of the top 20 cited websites to more accurately identify what types of sites were earning citations in the AIO.
The Top 20 Websites Have 72% of Total Citations
The top 20 websites (technically 21 with ties) have captured 72% of all mental health AI overview citations (3,504/4,812).
This absolute domination warranted further investigation, so we manually reviewed each site to identify the type of website (ignoring their TLD).
After reviewing each site, we grouped each of them into one of the following types:
- Gov: any government-run site (only nih.gov and nhs.uk ended up in the top 20)
- Affiliate sites: which we defined as any content site that earns revenue via affiliate partnerships, traditional ads, or sponsorships.
- Org: only actual non-profit organization websites (some .orgs are actually for-profit sites)
- UGC: user-generated content site, which ended up only being youtube.com
- Directory: one therapist directory (psychologytoday.com)
- Online Therapy Service: websites where people can pay for online therapy sessions
These categories give us the following data:
| Type of Website | Percentage of Total Citations | No. of Citations | No. of Sites |
|---|---|---|---|
| Affiliate Site | 36.7% | 1768 | 9 |
| Gov | 15.6% | 752 | 2 |
| Org | 12.5% | 601 | 6 |
| UGC | 3.2% | 157 | 1 |
| Directory | 2.5% | 118 | 1 |
| Online Therapy Service | 2.2% | 108 | 2 |
Surprisingly, affiliate sites were the most frequently cited in AIOs, capturing a whopping 36.7% (1,768/4,812) of total citations.
The top cited website was a government site (nih.gov) with 12.5% of total citations. However, the top 5 cited websites are rounded out by 4 affiliate-style, for-profit content sites (like WebMD).
This leads us to conclude that while certain high-authority government or non-profit websites may be uniquely effective at capturing AI overview citations, affiliate content sites with high domain ratings are even more likely to be cited within AI overviews.
- This leaves us with a few important questions:
- Who is writing the content for these sites?
- Who is reviewing that content?
- What editorial guardrails are in place for this content?
- What are these sites’ AI policies?
Do Keyword Rankings Matter?
In most cases, if a website was cited as a source in an AI overview, that website also ranked in the top 10 organic search results for the same keyword. When looking at the top 20 cited websites, if they are listed as a source in an AI overview they also ranked in the top 10 search results 78% of the time on average.
| Website | Cited and Ranks Top 10 |
|---|---|
| nih.gov | 66.9% |
| healthline.com | 81.1% |
| choosingtherapy.com | 79.5% |
| medicalnewstoday.com | 82.6% |
| webmd.com | 77.7% |
| verywellmind.com | 84.1% |
| clevelandclinic.org | 85.4% |
| mayoclinic.org | 78.0% |
| youtube.com | 0.0% |
| www.nhs.uk | 75.0% |
| psychcentral.com | 72.3% |
| psychologytoday.com | 55.3% |
| verywellhealth.com | 80.2% |
| mind.org.uk | 77.4% |
| helpguide.org | 72.1% |
| talkspace.com | 69.5% |
| psychiatry.org | 91.5% |
| goodrx.com | 66.0% |
| hopkinsmedicine.org | 85.1% |
| wikipedia.org | 86.4% |
| betterhelp.com | 70.5% |
A noteworthy outlier to this trend is YouTube. YouTube was an AIO source for 13.1% of mental health queries, but did not rank in the top 10 search results for any of those keywords. This may not be as surprising as it first seems given the fact that Google seems to prefer to present users with written content for mental health queries unless the searcher filters for video content.
In addition, there was a strong correlation between the number of ranking mental health keywords for a website.
It’s interesting to see such a strong correlation between keyword rankings and being an AIO source in the mental health vertical considering that 80% of websites cited for ecommerce queries do not rank well.
Do AI Overviews Contain Dangerous Information for Mental Health Searches?
A major concern we had when we saw AI overviews being presented to users who were Googling topics related to mental health was whether or not dangerous information would be presented to people. This was not an unfounded fear since AI overviews had presented clearly wrong and potentially dangerous information for other queries. A few examples include:
- Recommending drinking urine to pass kidney stones
- Explaining the health benefits of running with scissors and taking a bath with a toaster
- Recommending adding glue to pizza cheese
- Recommending eating at least one small rock a day
So far we have manually reviewed 75 AI overview results for mental health keywords, such as “ocd symptoms in women”, “treatments for schizophrenia”, and “how to overcome social anxiety.” Outside of a few small word choices and phrasing issues, we did not notice any glaringly inaccurate answers or dangerous information being presented.
That being said, our reviews of individual AI overviews continue and we will update this section in the future should we discover anything of importance.
Future Analysis
Going forward we plan to take a deeper look at the following topics and will update this article accordingly:
- How query syntax affects whether or not a keyword has an AI overview
- How query syntax affects AIO sources
- How AI overviews have affected the traffic of websites that are commonly sourced in AI overviews
- What SERP features show up most often alongside AI Overviews
If there’s anything else you’d like to know about AI overviews in the mental health space, or you have any questions about the information presented in this article, feel free to shoot me an email at [email protected]
Definitions & Data Sources
The following section will define various terms used throughout the article and outline the data sources used in the analysis.
AI Overview Data
All data for AI overviews including the websites cited as sources, the number of AI overviews in mental health search results, and how often each website is cited as a source comes from Ziptie.dev.
Below are explanations of what the various AI overview-related terms used in this article and our graphs mean:
- AIO Source Frequency: this number is based on the total search results that have an AI overview (978). Specifically, this is the percentage of search results in which each website is cited as a source. For example, NIH.gov is a source in 41.7% of AI overview results for mental health searches. Please note that many sites are cited more than once in the same AI overview, but this number only looks at the proportion of AI overview search results a given website is present in.
- Percentage of Total AI Overview Citations: of the 978 keywords that result in an AI overview, there are 4,812 total source citations. This number is the percentage of the total citations each website or type of website has. This number does account for a website showing up as a source multiple times for the same keyword’s AI overview.
- Queries Ranking in the Top 10: Out of the 3,720 mental health keywords we’re tracking, this is the number of those keywords that a given domain ranks in the top 10 organic search results.
