SearchLift

Technical

How to optimize a Webflow site for AI search

Webflow puts the settings that decide what ChatGPT, Claude and Perplexity can read on four different screens, two of them behind a paid plan, and all four behind a publish. Here is the pass, in order.

Published
September 30, 2026
Reading time
9 min read
A table of four Webflow settings. SEO title and description live in Page settings, SEO tab, with no plan stated, and are live only after a publish. Robots.txt rules live in Site settings, SEO, Indexing, with no plan stated, live only after a publish. JSON-LD for one page lives in Page settings, Custom code, needs a paid Workspace or Site plan, live only after a publish. Code on every page lives in Site settings, Custom code, needs a paid Workspace or Site plan, live only after a publish.
Four screens, two plan gates, and one publish step that applies to all of them.

Two of the four controls are behind a paid plan

  • Page settings, SEO tab. The title and the meta description for one page. No plan stated on the help page.
  • Site settings, SEO, Indexing. The robots.txt rules for the whole site.
  • Page settings, Custom code. The head slot for one page. This is where JSON-LD goes. Paid Workspace or Site plan.
  • Site settings, Custom code. Head and footer code for every page. Paid Workspace or Site plan.

Robots.txt is one screen, and the tokens are the whole job

TokenWhat it is forDisallowing it
The five tokens worth a line each on a business site. Behaviour quoted from each vendor’s own documentation; the full set, with the user action fetchers, is in the robots.txt article linked below.
txt
User-agent: OAI-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: GPTBot
Allow: /

User-agent: Google-Extended
Allow: /

Sitemap: https://example.com/sitemap.xml

Structured data cannot arrive through the Data API

Two routes compared. Route one, the Data API: an authorised app, then the Pages endpoint which writes the SEO title and description, then no per page head slot, so the JSON-LD does not arrive. Route two, Page settings: the custom code field inside the head tag, then the JSON-LD block up to 50,000 characters, then publish, after which the markup is live and readable.
Schematic. Both routes end at the same page, but only Page settings can carry markup.
html
<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "Organization",
  "name": "Your Company",
  "url": "https://example.com",
  "logo": "https://example.com/logo.png",
  "sameAs": [
    "https://www.linkedin.com/company/your-company"
  ]
}
</script>

Webflow’s own AEO tools are sold to Enterprise

Webflow, AEO feature page, read 29 September 2026

The pass, in order

  1. 01Open Site settings, SEO, Indexing. Read the current robots.txt. If any line disallows OAI-SearchBot, PerplexityBot or Claude-SearchBot, that is the first thing to fix. Save and publish.
  2. 02Pick the one page that answers the buying question your customers actually type. Not the homepage. In Page settings, SEO tab, rewrite the title so it states the answer, and the meta description so it summarises it in a sentence a person would read aloud.
  3. 03In Page settings, Custom code, paste the JSON-LD for that page into the field inside the head tag. Organization on the homepage, Product or Service on a commercial page, FAQPage only where the questions and answers are visible on the page.
  4. 04Publish the site, then confirm the change is live by fetching the published page rather than trusting preview. Run curl -s https://example.com/page | head -60 and read the title and the JSON-LD out of the response.
  5. 05Record what the assistants say about you today, before and after, so the change has a measurement attached to it rather than a feeling.
searchlift · console
The SearchLift audit screen listing pages with their issues and a fix action beside each one.
The audit lists the pages and the issue on each, so the pass above runs in priority order. Demonstration workspace.

Common questions

01How do you optimize a Webflow site for AI search?
02Does Webflow let you add structured data?
03Can you edit robots.txt on Webflow?
04Is Webflow AEO available on every plan?
05Why did my Webflow SEO change not show up?
Sourceshelp.webflow.comhelp.webflow.comwebflow.comdevelopers.openai.com