Apple On The Agentic Web: Every Way It Uses Your Website

Remember Apple Intelligence? Apple probably doesn't want you to. In June 2024, a Siri that was going to be "more natural, more contextually relevant, and more personal" was announced. Apple confirmed in March 2025 that it would take longer than it thought, and it arrived on 14 September 2026 under a different name, Siri AI, running on Apple Foundation Models built on Google's Gemini. Apple's own words in the joint statement: Google's technology "provides the most capable foundation for Apple Foundation Models".

That reads like a failure but I think it is the opposite. Apple is good at two things, hardware and making a system feel like one thing, and neither of those two things is training a frontier model. Paying for the model and keeping the integration may turn out to be the smartest call anyone has made about AI on phones.

The integration is the part nobody else can copy. ChatGPT, Claude and Perplexity run as apps on somebody else's operating system. Apple and Google write the operating system. Apple and Google can put an assistant inside the phone. They can build a Samantha!

For a website, that matters. Applebot has been crawling for years, feeding Spotlight, Safari and Siri, and it still is. What is new is how many ways someone can now reach your pages without opening a browser: Siri answering out of the index, Safari watching a page and reporting back when it changes, Siri reading whatever is already on the screen. Two of those three have no user agent, and Apple documents no control over either.

None of it changes what you should do. It makes it more urgent. More people are going to reach your website without looking at it, through more kinds of software, and the ones that can read it will use it.

Contents

• Two Apple Crawlers Fetch Your Pages, And A Third Is Undocumented
• Siri Answers And Onscreen Awareness Reach Your Content Without Touching Your Server
• Apple Publishes Two AI Opt-Outs, One For Training And One For Answers
• Apple's AI Opt-Outs Do Not Remove You From Apple Search
• Applebot-Extended, nosnippet And noindex Control What Apple Does With Your Page
• Siri AI Reaches iPhone 15 Pro And Newer, And Not In The EU
• Three Things People Get Wrong About Applebot-Extended
• Two Things To Check On Your Own Website

Two Apple Crawlers Fetch Your Pages, And A Third Is Undocumented

Apple documents two agents that make requests to your server. A third started making them on 14 September 2026, and Apple has not named it.

Applebot

• What it is: Apple's search and indexing crawler
• User-agent: Mozilla/5.0 (Device; OS_version) AppleWebKit/WebKit_version (KHTML, like Gecko)Version/Safari_version [Mobile/Mobile_version] Safari/WebKit_version (Applebot/Applebot_version; +http://www.apple.com/go/applebot)
• What it feeds: "the search technology integrated into many user experiences in Apple's ecosystem including Spotlight, Siri, and Safari", plus training for Apple's foundation models, plus "additional context and up-to-date content" for generated answers
• Respects robots.txt: Yes
• Verification: Reverse DNS within *.applebot.apple.com, or CIDR match against Apple's published Applebot CIDR list (24 IPv4 prefixes, checked 2026-09-17)
• Primary source: Apple, About Applebot (Published Date: September 04, 2026)

Applebot is the one Apple agent whose identity can be checked rather than believed. Reverse DNS resolves inside applebot.apple.com, and Apple publishes the address ranges as a file you can fetch. Anything can send Applebot's user-agent string. Only Apple can send it from Apple's addresses.

iTMS

• What it is: Apple's Podcasts crawler
• What it reaches: "only crawls URLs associated with registered content on Apple Podcasts"
• Respects robots.txt: No. Apple, verbatim: "The iTMS user agent does not follow robots.txt, as it is not a general search crawler."
• Verification: Not documented separately from Applebot's methods
• Primary source: Apple, About Applebot

iTMS ignores robots.txt, and Apple says so rather than leaving you to find it in a log. If you publish a podcast feed, iTMS fetches the URLs registered with Apple Podcasts whatever your robots.txt says.

The fetcher behind Notify Me

• What it is: Safari monitors a page you choose and alerts you when it changes. Apple: "It can also notify users of website changes, like product restocks or price drops, with Notify Me"
• User-agent: None documented.
• What it reaches: Any page a user points it at, repeatedly, for as long as the alert is active.
• Respects robots.txt: Unknown. Apple does not say.
• Verification: Not possible. Whether the request comes from the user's device or from Apple's servers is stated in neither Apple's crawler documentation nor its Notify Me announcement, both read 2026-09-17.
• Primary source: Apple Newsroom, 14 September 2026

This is not really a crawler. It is an agent with a crawler as a tool, watching one page on somebody's behalf and reporting back when the price moves. Something is going to start checking one of your product pages, over and over, and Apple has not said what it calls itself or where it comes from. That is the whole of what is published, and anyone telling you how to prepare for it is guessing.

Siri Answers And Onscreen Awareness Reach Your Content Without Touching Your Server

Siri's answers and onscreen awareness never appear in your server logs, and for different reasons. Siri answers are built from pages Applebot fetched earlier, so the request happened weeks ago and not at the moment anyone asked. Onscreen awareness makes no request at all, because the page is already open in front of the person asking.

Siri AI answers

• What it is: The assistant released 14 September 2026, in beta
• Where the content comes from: "Applebot crawled data may be used to provide additional context and up-to-date content when AI models are used to generate output... For example, answering broad world knowledge questions in Siri and Search that may include links to sources and websites used to help generate the answer."
• The model underneath: Apple Foundation Models, which per a joint Google and Apple statement of 12 January 2026 "will be based on Google's Gemini models and cloud technology"
• Publisher control: nosnippet for the answers, Applebot-Extended for training, neither for presence
• Primary sources: Apple newsroom 2026-09-14, and Google and Apple joint statement, 12 January 2026

Apple's control covers its own foundation models. Those models are built on Gemini, per a joint statement from both companies in January. Whether disallowing Applebot-Extended reaches the model underneath or only the layer Apple adds is not addressed in either company's documentation.

Onscreen awareness

• What it is: Siri reads the page already open on the user's screen. Apple: "while viewing their favorite sports team's website, a user can ask Siri to find all the upcoming home games and add them directly to their calendar"
• Does it fetch: No. The browser already loaded the page.
• Publisher control: None. The page was loaded by the person reading it.
• Primary source: Apple newsroom 2026-09-14

Nothing is wrong here. A person opened your page in a browser. Your server was asked for it once, by them. What their phone does with the page after that is between the person and their phone. Your server never hears about it, because nothing else is asked of it.

Apple Publishes Two AI Opt-Outs, One For Training And One For Answers

Apple documents fetching and use as two separate events with two separate control sets. Google draws the same line with Google-Extended. What Apple adds is a second control, so training and answers are two decisions instead of one.

On Applebot-Extended, Apple: "Applebot-Extended does not crawl webpages. Webpages that disallow Applebot-Extended can still be included in search results. Applebot-Extended is only used to determine how to use the data crawled by the Applebot user agent."

Every control Apple publishes governs what it does with a page it already holds. That reaches Siri's answers, which come out of the index Applebot built. It does not reach the other two. Nothing you can set governs a page already open on somebody's screen, and Apple has not said whether the Notify Me fetcher reads robots.txt at all.

Apple's AI Opt-Outs Do Not Remove You From Apple Search

Apple's crawler page states that content stays discoverable in Spotlight, Siri and Safari even with both AI opt-outs applied:

"Even if you disallow Applebot-Extended and tag website content with the nosnippet meta tag, your website instructions may still allow Applebot to crawl your webpages. Your content will remain discoverable through Spotlight, Siri, and Safari, as well as other system-wide features on Apple devices."

Read quickly, that sounds like Apple admitting its controls do not work. It is narrower and fairer than that. Applebot-Extended is an AI training opt-out, nosnippet is an AI answers opt-out, and neither is a search opt-out. Staying discoverable in Spotlight, Siri and Safari is the index, which existed before any of this and has its own control, which is Applebot itself.

One control over Applebot is not Apple's. Since 15 September 2026, Cloudflare's Block setting covers what it calls mixed-use crawlers, "including Applebot, Bingbot, and Googlebot", so selecting it stops Applebot reaching your website at all, search included. That is a lever at your own CDN rather than in Apple's documentation, and it arrived the same week Siri AI did.

In Apple's own documentation, the only complete exit is disallowing Applebot. It does three jobs at once: it builds the search index, it supplies training data, and it provides live context for generated answers. Most people running a website would want to decide on those separately. One directive covers all three, and using it costs you Apple search entirely.

Applebot-Extended, nosnippet And noindex Control What Apple Does With Your Page

Apple publishes three directives that govern what it does with a page after Applebot has fetched it. Two of the three are AI controls and one is not.

Applebot-Extended

• What it is: A robots.txt token. Not a crawler.
• Apple's own words: "Applebot-Extended does not crawl webpages. Webpages that disallow Applebot-Extended can still be included in search results."
• What it controls: "opt out of their website content being used to train Apple's general purpose foundation models powering generative AI features across Apple products, including Apple Intelligence, Services, and Developer Tools"
• What it does not control: Search inclusion, Siri answers, Spotlight, Safari
• Ranking effect: None, per Apple: "Site rules for Applebot-Extended are not considered in ranking for Search."
• Primary source: Apple, About Applebot

User-agent: Applebot-Extended
Disallow: /

This is the control Cloudflare accepted as Apple's training opt-out, and training is all it covers. Your pages stay in search, in Spotlight and in Siri. Apple also puts a thumb on the scale in the same section: "Allowing Applebot-Extended will help improve the capabilities and quality of Apple's generative AI models over time."

nosnippet

• What it is: A meta tag, originally a Google directive, honoured by Apple
• What it controls: "Web publishers can opt out of their content being used in these broad world knowledge answers by applying the nosnippet meta tag to specific content." Also: "Applebot won't generate a description or web answer for the page."
• Granularity: Page level only. Apple: "This signal applies at the page level. Section-level markup using hasPart is not supported."
• What it does not control: Training, crawling, search inclusion
• Primary source: Apple, About Applebot

This is the only published control over what Siri says about your page. Applebot-Extended governs training and does not touch it. nosnippet is the one for answers, and it works a page at a time.

noindex

• What it is: A meta tag directive
• What it controls: Apple: "Applebot won't index this page, and it won't appear in Spotlight or Siri Suggestions."
• What it does not control: Crawling, training
• Primary source: Apple, About Applebot

noindex is the blunt instrument, and it removes the page from Spotlight and Siri Suggestions. Suggestions are a different surface from the generated answers nosnippet covers, and Apple documents them separately.

Siri AI Reaches iPhone 15 Pro And Newer, And Not In The EU

Siri AI requires an iPhone 15 Pro or newer, and Apple has not released it in the EU on iOS. The device list and the country list are both shorter than the launch made them sound.

• Hardware. Apple's eligibility list, published 14 September 2026: "iPhone 16 models or later, iPhone 15 Pro, iPhone 15 Pro Max, iPhone Air, iPad mini (A17 Pro)", plus iPad and Mac with M1 or later, Apple Vision Pro, and Apple Watch Series 9 or later when paired with a compatible iPhone. An iPhone 13 Pro is not on that list, and no software update puts it there.
• Geography. Apple: "Siri AI will not be available initially in the EU on iOS, iPadOS, and watchOS." And: "Siri AI and the other new Apple Intelligence features will not be available in China while Apple works through regulatory requirements." macOS is out of the exclusion, which Apple stated directly in June: "EU users will be able to access Siri AI on macOS 27 and visionOS 27." The watchOS exclusion follows from hardware rather than regulation, per the same post: "Because Siri AI on watchOS 27 requires a paired iPhone with Siri AI, EU users will not have access to Siri AI on watchOS 27."

The same website answers differently depending on which country the reader is in and how old their phone is. Two people ask Siri the same question, one gets an answer built partly from your pages and one does not, and neither of them was asked.

Apple's reason is Apple's account of a live argument. In its June post it says the Digital Markets Act would oblige it to give any virtual assistant "the ability to read and send messages, make purchases, access files, and execute actions across any app", and to act "without a user's ongoing visibility and control". Apple offered an intermediary it calls Trusted System Agent plus an 18-month phased rollout, and says "The European Commission said no." There is no timeline.

Releasing Siri AI in the EU would have obliged Apple to open the same access to everyone else's assistant, and Apple chose not to. The part of its own case that does not fit is macOS. Apple released Siri AI on macOS and visionOS in the EU on the same day, so whatever the obligation is, Apple accepted it on one platform and refused it on another.

Three Things People Get Wrong About Applebot-Extended

1. Applebot-Extended is not a crawler and will never appear in your logs. It makes no HTTP requests. Apple states this in one sentence, on the same page that documents both.
2. Applebot-Extended is not the Apple AI opt-out. It is the training opt-out. The answers opt-out is nosnippet, which Apple documents in a different section of the same page.
3. Neither of them is a search opt-out, and Apple says so on the same page. Staying in Spotlight, Siri and Safari is the index doing what it has always done. The control for that is Applebot.

Two Things To Check On Your Own Website

Another category of software is reading your website now, in more shapes than it did in August. That is the same work it has always been: a price that is a number rather than a picture of one, a catalog reachable without running your JavaScript, a checkout that finishes without a hover.

Check which control you actually have set. Open your robots.txt and look for Applebot-Extended, then view-source on a page that matters and look for nosnippet. Those govern different things, and having the first without the second means you opted out of training and left the answers alone. That may be what you meant. It is worth knowing it is what you did.

Stop trusting the user-agent string on its own. Anything can send Applebot's user agent. Apple publishes 24 IPv4 prefixes and reverse DNS in *.applebot.apple.com, so a claim to be Applebot can be tested rather than taken. Check it before you decide what any Apple-labelled traffic in your logs actually was.

Related

• The AI User-Agent Landscape in 2026: A Complete Reference. The parent registry. This page is the Apple deep dive.
• The Web Started Checking ID, And 119 Of 153 AI Crawlers Cannot Prove Who They Are. Apple is in the minority that publishes verifiable identity.
• Cloudflare Will Write Your robots.txt, And It Has A Point. How the CDN-side controls over crawlers like Applebot work.