ChatGPT Search Vs. Google Vs. Bing Search Results

The function of AI in search engine marketing has stored quickly evolving over the previous few months.

Google formally launched AI Overviews in Could, and Bing launched its generative search pilot in July, with a full launch in the beginning of October. On July 25, OpenAI introduced a SearchGPT prototype. Then, on October 31, it launched ChatGPT Search.

OpenAI are calling the brand new product, which I’ve examined for this text, ChatGPT Search, however elements of the business are already beginning to use the names ChatGPT Search and SearchGPT interchangeably, as they’re successfully one and the identical.

Each the Google and Bing AI launches, alongside the rise of multi-modal search (e.g., TikTok), have been impactful, however the ChatGPT Search launch feels completely different.

ChatGPT already drives site visitors to web sites, and as of August 2024, it reported 200 million weekly common customers (WAU) – that is all whereas AI is gaining momentum and person belief, with skepticism nonetheless a barrier to mass adoption.

In February 2024, Gartner predicted that “search engine quantity” will drop 25% by 2026.

Having performed with SearchGPT, this prediction feels prefer it may turn out to be a actuality.

ChatGPT Search Commentary Abstract

• SearchGPT will (and may) cite webpages that don’t rank within the prime 100 basic search outcomes of Bing. This raises the significance of understanding the distinction between listed and ranked.
• There are discrepancies between Bing’s personal generative outcomes and the SearchGPT responses. As a rule, these are substantial.
• A single area can have a number of pages cited inside a single response.
• SearchGPT’s response for a similar question can fluctuate, even when the search is comprised of the identical ChatGPT account and IP.
• On some area of interest queries I examined which have restricted data on-line, SearchGPT was extra correct than each Google and Bing.
• The native search expertise feels missing, and for a variety of queries, not reliable.
• Some SearchGPT maps will be populated wholly by outcomes from a single supply, and the variety of “map outcomes” tends to lower the extra sources are included.
• Some cited hyperlinks from SearchGPT have been appended with the parameter ?utm_source=chatgpt.com when clicked on. UTM monitoring is a key function.
• Rating increased in Bing doesn’t imply referencing higher in SearchGPT, however it’s extra carefully aligned than AI Overview information.
• Responses are closely text-driven. Some do set off a map (native searches) and others pictures – however these aren’t assured.
• Queries in trend and journey are inclined to set off pictures within the response forward of different queries when examined.
• The picture sources in carousels don’t seem like included within the citations sidebar.
• It’s potential for content material behind a paywall to be cited by SearchGPT.
• SearchGPT can cite a webpage that returns an lively 404 if you attempt to view it.
• SearchGPT displays a greater understanding of time-sensitive queries than AI Overviews, even when a time variable isn’t included within the question.

Rising Bing Market Relevance & Significance To ChatGPT Search Inclusion

Bing is quick changing into a search engine we have to change our views on. For so long as I can bear in mind, Bing has all the time been an afterthought to the business and has solely actually been distinguished in particular sectors and goal markets.

Whereas its direct market share won’t be rising to rival Google in that sense, the truth that Bing is being utilized by a number of giant language fashions (LLMs), and ChatGPT Search is not any exception.

OpenAI has been utilizing Bing for a very long time, so its inclusion in SearchGPT isn’t a revelation, however a reinforcement of Bing’s evolving place within the general “search” panorama. We have to cease simply evaluating search engines like google and yahoo and their direct market share.

Which means that web sites (and webpages) not listed by Bing is not going to seem in ChatGPT Search.

Anecdotally, site visitors from LLMs to all web sites I’ve GA4 entry to has elevated exponentially over the previous three months.

Of all LLM site visitors, ChatGPT accounts for 31% of my web site (year-to-date), with it as a lot as 60% on some web sites alongside Perplexity, Claude, Copilot, and so on.

Breakdown of LLM site visitors (12 months up to now). GA4 information. Looker Studio visualized (Picture from writer, November 2024)

Nevertheless, this doesn’t essentially imply it’s good to rank excessive in Bing to be included. Very like Google’s AI Overviews, ChatGPT selects sources outdoors of the top-ranking outcomes.

It’s additionally price noting that whereas Bing is essential to the SearchGPT ecosystem, it isn’t the one supply.

Evaluating SearchGPT, Google, And Bing Search Outcomes

To see how the search leads to ChatGPT evaluate to each Bing and Google, I’ve checked out a variety of queries throughout Native, Your Cash or Your Life (YMYL), ecommerce, and a few informational/time-sensitive searches that I’ve personally carried out over the weekend.

I’ve tried to reside with ChatGPT seek for 48 hours moderately than subjecting it to a barrage of random queries.

I’ve not recorded all the queries on this article, however I’ve included the important thing ones in addition to some oddities and variations.

The variety of webpages referenced that don’t seem within the prime 100 basic ranks of Bing actually highlights that different sources are at play right here, and that optimizing for Bing doesn’t imply optimizing for ChatGPT and different LLMs straight.

In AI Overviews, when a webpage is referenced outdoors of the highest 100 basic outcomes, I really feel we simply assume it’s come from Google’s general database. In any case, Google crawling and constructing a database(s) of pages on the web is understood – however may OpenAI be doing the identical?

ChatGPT Search “Maps” & Native Search

Native queries are essential to a variety of companies. SearchGPT (on the time of writing) has a considerably restricted native search expertise.

When trying to find [enterprise technical seo agency], it supplied me with three outcomes: Two near my IP deal with and the third in a random location within the South of England.

Picture from writer, November 2024

There are a handful of “company hotbeds” within the UK, and a variety of companies in my native space. So, if SearchGPT have been going off IP, it will have included extra companies regionally. To then stretch the online additional and ignore London within the outcomes for broader outcomes doesn’t make sense.

Whereas not one of the three suggestions have been improper, SearchGPT appeared to hallucinate on the second end result.

Within the first and third outcomes, it added the label “Promoting Company,” however the second end result was given the label “Telecommunication Service.”

I’ve hung out trying by means of the company’s web site, social profiles, backlink profiles. I’ve crawled the web site with varied customized path extractions setup. I’ve checked out all of the schema markups that have been carried out. I can not fathom or discover a purpose as to why it has been categorized as a telecommunications service.

When repeating the seek for [digital marketing agency in new york], it gives the map, however with eight outcomes all from the identical supply.

Picture from writer, November 2024

That is much less helpful because of this, as being a reference for this question is principally a sport of showing in a sure web site checklist – which is a paid membership solely.

The web site used to compile all of the map outcomes ranks No. 4 in Bing classically, so it’s no less than a high-ranking end result – however this simply seems like scraping an inventory on a web site and is a lazy end result.

Triggering The Map

The map end result (as proven within the screenshot above) doesn’t all the time set off.

For a variety of queries after I specified a location, it simply supplied the checklist outcomes with out the choice to view a map. That is completely different from the “checklist” toggle on the Map, as this retains the map however creates an inventory beneath it (which is similar to the Google Map Pack).

Talking of Google Maps, should you ask SearchGPT explicitly to point out you the outcomes on a Map, it directs you to Google Maps:

Picture from writer, November 2024

YMYL Searches

YMYL queries refer to look queries that would doubtlessly influence an individual’s well being, security, or monetary stability.

It’s extensively understood (and communicated) that Google and Bing place higher emphasis on these queries and make sure that correct data is supplied from reliable sources.

“Remineralizing Gum”

In line with Glimpse, “remineralizing gums” noticed exponential search curiosity in October 2024.

As that is an rising question within the well being area, one may assume that the development shall be picked up by web sites promoting the product (or adjoining merchandise) and producing content material across the matter.

SearchGPT feels prefer it performed it secure with this question, solely using WebMD and Wikipedia as citations earlier than offering references to quite a lot of different web sites.

“How To Decrease Ldl cholesterol”

The response for this question doesn’t spotlight something harmful or out of the unusual and is measured in response.

I additionally acknowledge a variety of the web sites within the citations, which assist with trusting the data.

Apparently, SearchGPT is pulling from two separate URLs on the Mayo Clinic web site, regardless of one among them not rating within the prime 100 for Bing.

“Who Can Signal A Contract On Behalf Of A Firm”

On the primary run of the question, the response contained a extra textbook reply, citing the 4 most important standards underneath English Regulation for a contract to be legitimate (Supply, Acceptance, Consideration, Intent), in addition to two extra bullet factors round capability and that the contract itself is authorized.

Picture from writer, November 2024

From this response, the cited supply URLs and prime referenced URLs barely ranked within the prime 100 outcomes of Bing:

Nevertheless, after I carry out the identical question on a unique pc – on the identical account and similar web connection – I get a very completely different response:

Picture from writer, November 2024

The citations and search outcomes supplied additionally differ, with solely one of many authentic quotation sources remaining.

On the second run of this question, even fewer citations ranked within the prime 100 Bing outcomes.

Apparently, when in search of the domains on the second information pull for this question in Google, the domains ranked increased with completely different URLs than these cited by ChatGPT Search.

“How Many Credit score Playing cards Ought to I Have?”

Testing monetary queries, I noticed the closest alignment to each Bing and Google’s first web page of outcomes.

4 of the 5 cited sources within the desk above seem on Google’s first web page (within the UK) for the question, with the one exception being the financial institution.marksandspencer.com URL, which seems in the course of web page two on Google.

Ecommerce Searches

Transactional searches in ecommerce check with queries the place the person intends to make a purchase order or full a transaction.

These are sometimes high-intent searches, the place the person is actively in search of a services or products, they usually’re shut to creating a shopping for choice.

In my fashions, I’ve talked about how AI can’t essentially fulfill the complete extent of an ecommerce question (as AI can’t facilitate the buying and buy expertise); I’ve classed this person group as “Purchasers.”

Dan Taylor’s Mannequin of AI Affect on Person Varieties (Picture from writer, November 2024)

The easier and extra open the question, the extra seemingly AI can steer and affect, however as a person narrows right down to make a purchase order (strikes from being a client to a purchaser), it nonetheless wants to have interaction with manufacturers and ecommerce web sites.

“Finest Christmas Items For Him 2025”

One other optimistic level for SearchGPT is that all the webpages referenced and cited for the question have been revealed (or no less than declare to have been revealed) previously month.

An adjoining take a look at I’m operating on AI Overviews for the time being for related queries has content material way back to 2017 being referenced and cited in AI Overviews.

“Black Friday Offers”

Picture from writer, November 2024

Whereas being very single-source heavy, the second quotation for [Tips for maximizing Black Friday Savings] got here from an Australian web site.

Google’s AI Overviews have obtained some criticism on X (Twitter) and different platforms from a variety of search engine marketing professionals as to the inclusion of Australian sources in non-AU markets.

This could possibly be a sign that SearchGPT might exhibit the identical habits throughout queries.

“Nike Air Max Dimension 10 Offers”

Simply trying to find merchandise in SearchGPT appears to be interpreted as informational, however including a modifier as “offers” brings in regards to the industrial intent and adjustments the response, whereas sustaining the conversational tone:

Picture from writer, November 2024

This end result additionally caused some hallucinations/misinformation.

On the Nike supply cited:

• I couldn’t discover a pair of Air Max ’90s for £115.99 – they have been all cheaper.
• The one Air Max 97 on the cited webpage was a kids’s shoe, lowered to £52.49 – once more, quite a bit cheaper.

This isn’t splendid, as if I used to be trying to purchase a pair of Air Max ’90s at an inexpensive worth by means of Google’s PLAs, I can discover them for £80 (and a few cheaper relying on the mannequin). This can be a lot cheaper than what SearchGPT is saying Nike is promoting them for straight, so it may detract gross sales out of your model web site if this misinformation persists.

This question was one other occasion of Google rating URLs for the area, e.g., JDSports, for the question in place – however not the URL cited by ChatGPT Search.

Journey Searches

“Finest Vacation Locations 2025”

Very like a variety of queries within the journey sector, this triggered a picture pack above the carousel.

Picture from writer, November 2024

Having labored with a lot of journey firms and supported the business by means of occasions and the Institute of Journey & Tourism, I can actually say this can be a very eclectic checklist of locations to think about.

The citations are very publisher-heavy, with different journey firms (and journey blogs) within the search outcomes part.

“Luxurious Motels In Napa Valley”

To my shock, this question triggered the SearchGPT Map.

Picture from writer, November 2024

The Telegraph.co.uk end result was referenced twice for 2 completely different accommodations within the SearchGPT Map Pack.

Reviewing the suggestions and sources, there doesn’t seem like a logical connection as to why these sources and (particularly) these areas have been chosen.

Google’s lodge function is much more complete and interactive, so I don’t see this threatening the lodge search journey any time quickly.

“Abu Dhabi Grand Prix”

Sports activities tourism is rising in reputation, and F1 is a globally fashionable sport.

Trying to find particular races can be a tough question, which is what I wished to check out.

Trying to find “[location] grand prix” has a variety of frequent interpretations. It could possibly be an informational search, a navigational search, or with transactional intent and eager to analysis ticket costs.

Picture from writer, November 2024

I really feel like SearchGPT understood this, because the 5 web sites referenced as citations cowl these bases fairly properly, and it doesn’t lean into being too closely informational or transactional.

This can be a related end result to any Grand Prix question, e.g., Bahrain GP, Belgium GP, Singapore GP.

From the checklist of referenced webpages, the **P1Travel URL posed an anomaly throughout all the searches I’ve finished to attempt to break SearchGPT. The URL returns a 404:

Screenshot from p1travel.com, November 2024

Journey search engine marketing has been evolving for a variety of years now, and for my part, LLM platforms pose the best potential to disrupt this business extra persistently than others.

Different Question Observations

Over the previous 48 hours, I’ve made notes of queries made all through the day by means of Google and repeated them on Bing and SearchGPT.

“England V Samoa Rating”

Regardless of being on the match itself, what triggered me to take a look at this question was my spouse commenting on the rating – and the rating she quoted was improper.

So, I replicated her search on Google and located the under end result:

Screenshot for seek for [england v samoa score], Google, November 2024

Whereas the “High Tales” part does carry correct outcomes, Google’s function on the prime of the SERP brings the results of a sport over a 12 months in the past – in a unique sport.

Now, I can perceive why Google is falling down right here.

Rugby Union is the second hottest sport within the UK, and Rugby League is the eleventh hottest. So, selecting to point out this sport over the opposite could be acknowledging a higher frequent interpretation of the question, and trying to cater to a bigger potential search viewers.

I then wished to see how ChatGPT Search fared with the identical question:

Screenshot of seek for [england v samoa score], ChatGPT Search, November 2024

Whereas it isn’t as visually distinguished, the generative textual content cited from the BBC accommodates the right scoreline in addition to different factually right data.

The downfall then comes that the information tales being pulled by means of relate to a unique sport (that occurred the earlier week).

As I reviewed all the citations supplied by ChatGPT Search, I seen that it pulls by means of information tales referring to the identical incorrect match that Google cited prominently within the SERP function.

However the AI has been capable of distinguish the 2 and supply a well timed end result, doubtlessly displaying a higher understanding of queries that could be time-sensitive and the idea of QDF in opposition to leaning in the direction of the “extra frequent” dominant interpretation.

As Bing performs a distinguished function in SearchGPT, it’s no shock to see Bing additionally prioritizing the newer incident of England vs. Samoa by means of the Bing Information function, though Bing does then function the identical match from over a 12 months in the past within the particular sports activities function.

Screenshot of seek for [england v samoa score], Bing, November 2024

“Manchester United v Chelsea”

To check true timeliness round reside occasions, I’ve carried out (and in contrast) this search question in the course of the reside soccer sport.

I truly discovered the ChatGPT Search end result to be probably the most informational and helpful – outdoors of reporting the reside rating.

Screenshot of seek for [manchester united v chelsea], ChatGPT, November 2024

The ChatGPT overview gives a variety of data in a single snippet – together with broadcast data.

Should you seek for broadcast data on Google and Bing, you don’t get this depth of knowledge straight within the outcomes – and sadly, a variety of publishers bury this data deep in an article, even when that article is written (and headlined) to focus on the question particularly.

To see if ChatGPT Search would present the in-game rating, I attempted repeatedly with variations of “Manchester United v Chelsea reside rating|in sport rating|present rating|rating now.”

However regardless of Bing displaying the reside rating in a SERP function and a variety of sources being accessed by ChatGPT that comprise the data, all it will give was variations of the under response – refreshingly asking the person to go to the supply web site for the data.

Picture from writer, November 2024

“Cheese Not Sticking To Pizza”

This was a distinguished question that Google’s AI Overviews fell down on earlier this 12 months, drawing scrutiny from the mass media.

Picture from writer, November 2024

I’ve examined a variety of the queries that tripped Google up, and up to now, SearchGPT appears to keep away from hallucinations and complicated outcomes.

Do I Assume This Will Disrupt Google?

Google has been going through disruption because the mass market’s relationship with cell gadgets modified.

Shifting away from desktop and using apps and browsers to analysis, store, and be entertained opened the doorways for brand new platforms to compete for display time and a focus.

On a latest podcast I ran, John Mueller stated that when Google first began to push in the direction of mobile-first, a variety of search engine marketing professionals and site owners have been skeptical and vocal that customers wouldn’t shift from cell to desktop to view their web sites.

search engine marketing as a apply is barely 30 (or so) years previous – and in 30 years, how customers work together with companies has modified drastically.

AI platforms, akin to SearchGPT, will disrupt Google as adoption will increase and the “early majority” of customers begin to use LLM platforms each day for duties they in any other case would have engaged the broader Google ecosystem with.

We are able to draw some parallels to the Mannequin T, and when Henry Ford launched the automobile, it was seen as a novelty and never for the lots – with individuals seeing them as “sooner horses” than a revolutionary mode of transport.

As search engine marketing professionals, we will see the “revolution” a part of it in movement, however we should do not forget that to the mass market, a variety of AI is merely gimmicks akin to eradicating issues and folks from the background of pictures on telephones or utilizing issues like Circle Search.

SearchGPT will contribute to the general disruption, and it’d pioneer the “what’s potential, “very like Jeeves was a pioneer in mass market web search engine adoption.

Additionally, I don’t really feel this is able to be an entire reply if I didn’t ask SearchGPT straight if it’s going to disrupt Google:

Picture from writer, November 2024

Which Search Is Finest?

In testing SearchGPT felt as if it had a greater understanding round time delicate queries and used newer sources.

Eradicating visible expertise from the equation, SearchGPT is healthier as objectively you need latest content material if you’re trying to find data round holidays, items, merchandise for “now,” and never viewpoints from over a 12 months in the past.

For native search, ChatGPT Search nonetheless has an extended technique to go and falls wanting have been Google and Bing are.

For my part, data was higher in ChatGPT Search than in Google and Bing, however not on all queries. General I nonetheless really feel it’s behind Google’s AI Overviews and Bing’s Generative Search by way of reply queries behind text-driven informational outcomes – and the ChatGPT Search interface is a bit bit too indifferent from what we’re used to seeing from a “Search engine”.

Closing Ideas

Brett Tabke has referred to SearchGPT because the “remaining” search engine, and I agree with this.

Whereas SearchGPT presently affords sure benefits over Google and conventional search fashions, it’s nonetheless a piece in progress.

Nevertheless, its potential trajectory is among the key insights to remove from SearchGPT.

This growth signifies that person interplay with the web is evolving as soon as once more, and as search engine marketing professionals, we should adapt as an business.

The rise of platforms like TikTok highlights the shift towards multi-modal search, urging us to maneuver past a slender concentrate on the “engine” facet of our work.

The core ideas of search engine marketing, notably technical search engine marketing, will all the time stay related, as AI crawlers should crawl, establish, and entry content material equally to conventional search engine crawlers.

I imagine it’s time for us to regulate our search engine marketing terminology, shifting the dialog from rating to referencing, and realigning what the general success of an search engine marketing marketing campaign is.

And we lastly begin taking credit score for higher funnel visibility and touchpoints and should not measured solely on bottom-of-the-funnel conversions.

Fairly than asking the questions “How will we optimize for SearchGPT?” or “How will we optimize for LLMs?,” we have to be taking a look at our market and the completely different LLM platforms and asking questions like:

• How does our viewers work together with SearchGPT, and the way will SearchGPT cease our viewers leaving the ChatGPT ecosystem?
• What data does the person get in the event that they keep on this ecosystem versus what they might obtain “historically”?
• When (and if) the person leaves this ecosystem, what’s their new entry level in our perceived funnel? How properly tailored is our funnel for new-to-brand (NTB) touchpoints at this stage?

Extra sources:

Featured Picture: Roman Samborskyi/Shutterstock