People who work on GEO often make one mistake (we’ve made it ourselves): treating all AI as a single thing to optimize.
Today, here’s a more practical judgment: every AI assistant, when it goes online, is using its own search engine behind the scenes. So “a given AI doesn’t recommend you” really means “you don’t rank in its search engine — or you’re not even indexed at all.”
That judgment can save you a huge amount of wasted effort. We followed it through a round of checks yesterday, and the results were surprising — in one major vendor’s search engine, only 3 of our site’s 300-plus pages are indexed, and all three are old versions from several years ago.
First, let’s be clear: when an AI goes online, where is it actually searching?
When an AI assistant answers a question like “recommend a few companies,” it roughly takes two steps: first it searches a batch of web pages, then it summarizes an answer from what it found.
The key is the first step: it usually uses its own search engine, or one it partners with. So whichever search engine indexes you well and ranks you high is the one that makes you more likely to appear in the corresponding AI’s answers. This isn’t mysticism — it can be verified step by step.
The corollary is just as direct: if you’re not indexed in a given search engine at all, the corresponding AI will never recommend you, no matter what — not because it dislikes you, but because it simply can’t find you.
How to check: two searches, five minutes
No tools or permissions needed — you can do it all in a browser. For every search engine you care about, do these two things:
- Check the index count: type
site:yourdomaininto the search box. It will list how many of your pages that search engine has indexed. Compare that number with the actual number of pages on your site. - Check keyword rankings: search your single most important business term (for example, “city + industry + company”) and see whether you appear in the first two pages.
Don’t check only the biggest one. Run through the search engines behind every AI you care about — the gaps between them can be far larger than you’d imagine.
What our check turned up: in one major vendor’s search, site: returned only 3 results — while our site has 326 pages.
What’s even more telling is what those 3 results look like: the titles are still an old version we used long ago, and the content is old articles from 2021 and 2024. In other words, its index is stuck far in the past — it knows nothing about any of the content or optimization work we’ve done over the past year or two.
On the same day, the same website showed completely normal indexing in several other search engines.
Low index count? First rule out problems on your own side
If you find that one search engine indexes surprisingly little, don’t rush to conclude that “this one is no good.” First rule out three things on your own side, in order — this is exactly how we worked through it, one step at a time:
- ① Has its crawler ever been here? Check your server logs for the matching crawler identifier. What we found: it comes hundreds of times a day, with no problems — so it’s not that it “can’t get in.”
- ② Is it being blocked by your own defenses? Look at the status codes that crawler receives. Ours were a uniform 200 — it was never rejected.
- ③ Is the content it reads complete? Many people don’t know how to do this step — request your own pages using that crawler’s identifier and check whether the title, description, and key body content are present in what comes back. If your content depends on script rendering, the crawler may well be getting an empty shell.
All three checked out fine for us: the crawler can get in, it isn’t blocked, and the content is complete and readable. So the conclusion is clear — the obstacle isn’t crawling, it’s index updating.
Crawlable but not indexed: usually these three reasons
- It can’t get your “table of contents.” When a crawler enters your site, it first reads
robots.txtto find your sitemap. If the address listed there is broken, it can only crawl links level by level on its own — extremely inefficient, and new content is easily missed. We had just found and fixed this problem the day before — every address listed was dead, and the one that actually worked wasn’t written in at all. - No one is “voting” for your key pages. When few internal links point to a page, the crawler considers it unimportant. We checked: among the site’s 300-plus pages, only 3 link to our most important company profile page.
- You’ve never submitted it yourself. Most search engines have webmaster platforms where you can submit a sitemap and push new links. If you’ve never submitted once, you’re relying on it to discover everything slowly on its own.
The three corresponding actions
| Action | What to do specifically | Difficulty |
|---|---|---|
| Fix the sitemap entry | Open yourdomain/robots.txt and click through each Sitemap address in it. Any that opens as a web page rather than XML code is broken — have your tech team fix it |
A few minutes |
| Add internal links to key pages | Naturally link to priority pages such as the company profile and core services from related articles, the footer, and the navigation. Keep it natural — don’t stuff links in | Low |
| Submit via the webmaster platform | Register for or log in to the webmaster platform of the relevant search engine, verify site ownership, then submit your sitemap; afterwards you can keep pushing new links | Verify once, then usable long-term |
Order matters: fix the sitemap first, then submit. Otherwise you’ve invited the crawler in, but it still can’t get that table of contents.
One more note: many webmaster platforms support QR-code login with social media accounts, so you may not need to create a separate account or complete real-name verification. Verifying your site usually means placing a file in your website’s root directory or adding a snippet of code to the homepage. It takes one round of help from your technical team, a matter of minutes. This barrier is lower than most people assume.
Setting Expectations: This Is a Slow Variable
Once these fixes are in place, you won’t see results the next day. Search engines need time to re-crawl and re-index your site, usually weeks to months.
So the right approach is: after making the changes, note the date and your current indexed-page count, then repeat the same site: query once a month to see whether the number is climbing. If it rises, you’re on the right track and can keep going. If it hasn’t moved at all after three months, there are still obstacles you haven’t cleared, so go back and work through the three steps above again.
Don’t judge progress by whether AI recommends you. That’s the end result, shaped by too many factors, and it’s random. Use your indexed-page count as the intermediate metric: it is more stable, reacts earlier, and better reflects whether the work you’re doing is having any effect.
A Few Common Questions
Q: How do I know which search engine a given AI uses?
A: A simple approach is to look at the sources it cites. When answering with web access, it usually labels its sources, and after a few answers you can tell which sites and systems it favors. Also, AI assistants from major tech companies generally prioritize their own search engines and content from their own ecosystems; that is a reasonable inference.
Q: If my indexed-page count is low, can I just publish more content?
A: No. This is the most common misconception. If the obstacle is that crawlers can’t reach your directory, or that you’ve never submitted your site, then no matter how much you publish, they still won’t find it. Fix the channels first; talk about volume later.
Q: Is the number returned by a site: query accurate?
A: It’s an estimate, and it fluctuates from one check to the next. But it’s good enough for reading order of magnitude and trends: the difference between a site of 300-plus pages with only a handful indexed and one with 200-plus indexed is not a margin of error.
Q: How many internal links should I add?
A: Keep it natural. Add them where they are genuinely relevant, for example linking to your Company Profile page when the company is mentioned at the end of an article. Adding links for their own sake, or forcing them into unrelated places, can actually be flagged as manipulation.
Want to know how much of your site is actually indexed across the major search engines?
Siwenlide runs websites and SEO/GEO content-led lead generation for small and medium-sized businesses in Beijing and the surrounding area: we check indexed-page counts and keyword rankings engine by engine, verify from server logs whether crawlers are accessing your site properly, fix the “channel” problems such as sitemap entry points, internal link structure and webmaster platform submissions, then record your indexing trend month by month. Make sure the content you publish can actually be received before you talk about volume. Visit siwenlide.com (or start with our Company Profile) or call 400-686-2011 to talk it over.
