What AI search engines actually see on 3,877 B2B software websites

What AI search engines actually see on 3,877 B2B software websites

AI search can reach and read B2B software websites. The bigger block sits in the firewall. At least 35 of 3,517 sites with a readable robots.txt block an AI search bot from the homepage. That’s a small number. Then I asked ChatGPT to open 132 sites that a bot test had flagged. 34 blocked it …

Which AI crawlers to allow: GPTBot vs OAI-SearchBot, by Scott King

GPTBot vs OAI-SearchBot: Which AI Crawlers to Allow

ChatGPT’s crawler spent 34.82% of its fetches on pages that returned a 404. Claude’s spent 34.16%. Googlebot, over the same period, spent 8.22%. Those figures come from Vercel and MERJ’s December 2024 analysis of their network and nextjs.org. A third of one crawler’s budget hitting dead pages is not a content problem. It is a …

157 of 169 B2B homepage headlines would work on a competitor's site

157 of 169 B2B homepage headlines would work on a competitor’s site

All 204 readable homepages in my 232-company benchmark let AI search in. Here is what the headlines miss, and why the fix is a differentiator your best-fit buyer can check. Your buyers now meet you through a machine before they meet you on your site. ChatGPT, Perplexity, Claude and Google’s AI Overviews read your homepage, …

Stop Political Texts and Telemarketing Calls, a free 5-step checklist from thescottking.com. Your number never leaves your device.

Stop Political Texts and Telemarketing Calls: Your Free 5-Step Checklist

Enter your number and state. You’ll get a short checklist of every official Do Not Call list that covers you, the steps that cut down political texts, and the free spam blocker for your carrier. It takes a few minutes. It’s free. Your number never leaves your device. This form has no server behind it …

AEO vs GEO vs SEO: What Changes, and What It Costs You

AEO vs GEO vs SEO: What Changes, and What It Costs You

Of the top 100 organic pages across 10 sites, 49 received zero traffic from LLMs. That comes from a May 2026 Search Engine Land analysis of roughly 150,000 pages. Half the pages winning in Google were invisible in AI answers. That is the honest case for treating this as new work. It is also the …

AI Visibility Tools: What They Measure and What They Cannot

AI Visibility Tools: What They Measure and What They Cannot

On October 18, 2025, more than 85% of tracked brands lost visibility in ChatGPT on the same day. Profound measured it across millions of prompts. Average brand visibility fell 31%. None of those brands did anything to cause it. ChatGPT changed how it handles brands as entities, and the answer set contracted for everyone at …

How to Improve Brand Visibility in AI Search Engines

How to Improve Brand Visibility in AI Search Engines

ChatGPT’s crawler spent 34.82% of its fetches on pages that returned a 404, according to a December 2024 Vercel and MERJ analysis. Googlebot hit 404s on 8.22% of its fetches. A third of one engine’s crawl budget landing on dead pages is not a content problem. That number explains why so much AI visibility work …

Why ChatGPT Recommends Your Competitors Instead of You

Why ChatGPT Recommends Your Competitors Instead of You

You asked ChatGPT the question your homepage claims to answer. It named four other companies. Yours was not one of them. The model is not ignoring you. It has no basis to choose you. That distinction decides what you do next. One version of this problem is technical and gets fixed in an afternoon. The …