What are the five observable stages?
- Prompt interpretation and fan-out. OpenAI says one ChatGPT prompt may be rewritten into one or more targeted queries, followed by additional searches after initial results are reviewed.
- Candidate retrieval. Those queries retrieve URLs from available search providers or an admitted corpus. A domain or recency restriction narrows eligible candidates; it is not a URL-submission command.
- Exact-address safety. OpenAI's link-safety paper describes a dynamic allow-list based on exact URLs previously visited by an independent search crawler, after canonicalization.
- Content extraction. A selected URL may be read from cached content or fetched live. An admitted search result can still produce a cache miss or extraction failure.
- Answer and citation selection. Retrieval does not guarantee that the model will quote, cite or recommend the page.
The ordering above is a high-level evidence model, not a disclosure of OpenAI's private implementation for a particular request. The first and third stages are directly documented by OpenAI. Their relationship to the native observations is an inference, stated as such.
What did the native URL controls show?
| Control | Observed result | What it separates |
|---|---|---|
| Known Presenc article | Opened normally | Admitted exact URL |
| Invented Presenc path or novel query value | Unsafe before extraction | Same domain is not enough |
Known BlueprintAI vercel.app root | Opened normally | No blanket Vercel exclusion |
| BlueprintAI fragment, hostname case or explicit port 443 | Canonicalized to the known root | Expected address normalization |
Visibly linked BlueprintAI /blog | Content-cache miss | Past safety, failed later extraction |
| Google Play parameters reordered | Known URL opened | Parameter-order normalization |
| Google Play parameter value changed | Unsafe before extraction | Value change creates a different exact address |
| Complete unseen Quoted First URL in a prompt | Rewritten into topical search; no target or verified request | Supplying a URL did not force fetch in this interface |
New Quoted First article, root, robots.txt and sitemap opened directly | Each stopped as unsafe before extraction | The observed absence covered the host, not one malformed article path |
| Complete article URL searched literally, then opened again | Search returned unrelated admitted pages; the second open remained unsafe | Putting an address in query text did not add it to the independent exact-URL set |
Natural article question restricted to quotedfirst.com | No target result | A domain restriction narrowed admitted candidates; it did not submit the host |
Re-searching the BlueprintAI root after a fresh open retained its older two-week search-crawl label. That is further evidence that a search snapshot and a new extraction are separate states. The complete dated record is available in the JSON observations and CSV export.
What do Exa and Firecrawl clarify?
Exa and Firecrawl are useful comparison models because their public interfaces name operations that are easy to blur together. Quoted First does not claim that either company powers OpenAI.
- Exa: its public API separates
/search, which finds URLs in Exa's curated index, from/contents, which retrieves a known URL from cache or live-crawls it according tomaxAgeHours. Exa defines crawl date as the date it discovered a link. - Firecrawl: its
/searchendpoint discovers pages and may scrape the returned results, while/scrapeextracts a supplied URL with separate cache controls. Its/crawloperation starts from a known URL and discovers linked subpages.
The comparison supports a practical measurement rule: record whether a URL was returned, whether it was opened, whether the origin received a request, and whether it was cited as four separate facts.
What can a new site do?
- Keep the canonical commercial URL stable instead of publishing a new synonym route at every test.
- Return useful public HTML with a self-canonical URL, indexable directives and no authentication wall.
- Allow
OAI-SearchBotand make sure the CDN or firewall delivers the same useful response a reader receives. - Link the page from normal visible navigation and an accurate sitemap. OpenAI also says URLs may arrive through third-party providers or links found on other pages.
- Publish specific, supportable information that can satisfy the buyer's real constraints after the URL becomes retrievable.
- Measure fixed elapsed gates. A bot request is not an indexing receipt, and an HTTP-200 submission response is not a ranking.
/robots.txt at 06:59 UTC but no verified request to this article. That removes configured Vercel firewall policy as an explanation for the immediate miss; it does not prove that OpenAI discovered or admitted the URL.robots.txt and sitemap before extraction. Searching the full article URL did not return Quoted First and did not make a second open eligible. In unrelated domain-restricted controls, bare vercel.app pages were searchable, but the youngest visible crawl labels were about four to six days old; a one-day preference exposed no clearly same-day Vercel page. The exact article path had no verified crawler request through 07:17 UTC. One verified ClaudeBot request reached the stable alias's robots file at 07:13 and received a redirect; that is a policy-file event, not content delivery or proof of IndexNow causation. This is an observed sample, not an indexing SLA.Quoted First's unrelated searches consistently show that team size, industry, workflow, company stage and budget change the candidate mix. Impossible nonce terms can be relaxed. Specificity is therefore a post-admission relevance lever, not a command that forces an unseen host into search.
Frequently asked questions
Does putting a URL in a ChatGPT search prompt force a live fetch?
Not in the native interface tested here. The prompt was rewritten around the topic, the target was not returned or cited, and exact Vercel metrics recorded no verified page request in the completed window. This does not establish behavior for every ChatGPT account, API mode or future implementation.
Did Vercel Firewall block the tested page?
No configured project rule did. The read-only audit found no active WAF configuration, custom rule, IP block, system bypass or draft, and Attack Mode was off. The clean audit narrows the failure to discovery or admission outside those project controls; it does not prove anything about an upstream provider's private index.
Can searching a literal new URL make ChatGPT open it?
It did not in this native control. Two searches containing the complete Quoted First article address returned admitted pages about ChatGPT web search instead. A direct open before and after those searches produced the same exact-address safety rejection. The test distinguishes a query mentioning a URL from an independent crawler having observed that URL.
Does a GPTBot crawl put a page into ChatGPT search?
No receipt says that it does. In Quoted First's test, an earlier page body was fetched twice by verified GPTBot, yet native searches still omitted it roughly 24 hours later. See the complete GPTBot experiment.
Does IndexNow force ChatGPT-search inclusion?
No immediate advantage appeared in Quoted First's first four hours. ClaudeBot fetched both frozen roots; Bing checked the submitted arm's ownership key; OAI-SearchBot checked only the untreated arm's robots file; and neither entered native results. See the matched IndexNow experiment.
Can an optimizer guarantee inclusion?
No. A provider can improve access, clarity, evidence and measurement, but cannot force a third-party search system to admit, retrieve, rank or cite a page. Quoted First currently reports zero customers and makes no placement guarantee.