Jori Ford took a different approach at Tech SEO Connect. While other speakers gave us frameworks and theory, Frost showed us how to run experiments—and wasn’t afraid to share the ones that failed.
Her core message: SEOs are scientists, and right now we’re in a stone age. The AI landscape is a black box, and the only way forward is structured experimentation. She built test sites, streamed logs in real time, ran parallel tests, and came back with findings that validate—and extend—what we heard from Krishna Madhavan and Martha van Berkel.
Her new mantra, borrowed from the scientist August Kekulé: “I will listen to any hypothesis, but on one condition—that you show me a method by which it can be tested.”
The Stone World: Why We Have to Start Over
Frost framed her talk around the anime Dr. Stone, where civilization collapses and a scientist has to rebuild society from scratch using only his knowledge. Her point: that’s where we are with AI. SEO used to be sorcery, then it became a discipline with rules. LLMs? Not so much. We’re back to experimentation.
“We as SEOs can assume nothing,” she said. “Once Google started helping us, we could establish boundaries. We knew what was going on. Now we’ve got to test everything.”
The good news? Last year the industry was scared. This year we’re empowered—because AI can’t do its job without the index, and who built the index? We did.
The JoriCon Experiment: How Frost Tested AI Crawlers
Frost set up two fake anime festival websites—JoriCon and JoriFest—on her own droplets so she could monitor logs in real time. The goal was to understand how AI crawlers access and process information, particularly structured data.
She tested three approaches: embedded JSON-LD on the page, a raw endpoint, and routing (pointing the crawler to where she wanted it to go). Her hypothesis was that routing would win—she’d direct the bot to her structured data endpoint and it would follow instructions.
She was wrong.
“It skipped over my route,” she said. “I failed.” But in failure, she learned something important: the bot wanted efficiency. Because she had already embedded the structured data on the page, it captured that information quickly and never bothered with the route.
She tested with ChatGPT and Claude (excluding Google since we already understand how it uses the index). Both behaved the same way. Importantly, she focused on RAG bots rather than training bots because you can force RAG bots to visit specific URLs and observe their behavior.
What the Logs Revealed
Watching the real-time log stream, Frost discovered something fascinating: the bots are smarter than we might assume. When ChatGPT visited her site, it didn’t just fetch the page she pointed it to. It started spoofing different mobile devices. It tried to access index pages. It checked for WordPress paths and configurations to understand how the site was built.
“It’s not dumb,” she said. “It’s looking at how we standardize content. You know how they say ChatGPT seems to prefer blogs? Wouldn’t it make sense that it tries to figure out if you are one?”
This matches what we’ve been hearing about AI systems trying to understand content architecture, not just content itself.
Structured Data Is for Logic
Frost ran ten parallel tests, each repeated five times, to understand accuracy and consistency. Her key finding reinforced what Krishna Madhavan said earlier in the day: structured data enables reasoning.
When she fed the bot simple, flat text (just paragraphs), it could retrieve the information. But when she asked complex questions—like “Which is cheaper for me to attend at this time, JoriFest or JoriCon?”—it couldn’t reason through dates, costs, and logistics.
When the same information was structured with JSON-LD, the bot could answer complex comparative questions accurately. Structure isn’t about whether the bot can find your information—it’s about whether it can reason with that information.
“More structure means more reasonable,” Frost said. “Schema is the API for logic.”
Discovery: How to Get Into Training Data
One of Frost’s most interesting experiments addressed discovery. Her test sites were new and not indexed. When she asked ChatGPT about JoriCon without providing a URL, it said it couldn’t find anything. It tried searching the index, couldn’t find it, and gave up.
But when she fed it the URL directly, something interesting happened. After the RAG bot processed the structured information and “trusted” it, the training bot came back to crawl the site.
“That was my aha moment,” she said. “You can actually feed the URL in, ask it to process that information. The more structured and clear the information, the more it trusts you. Once it trusted me, it came back and tried to train on me.”
This is significant for anyone trying to figure out how to get into AI training datasets. Trust comes first, built through structured, fresh content. Then the training crawlers follow.
Context Bias Is Real
Frost warned about context bias—the way AI systems cache information within a session and let that influence subsequent responses. Once she fed a URL to ChatGPT, it kept returning to that URL for information throughout the session.
The problem: if the first touch of information was unstructured, the bot used that cached version even after structured data was available. “It got dumber,” she said, “because it couldn’t logically answer me. I couldn’t add the structure in later because it had already started its session.”
The practical implication: when testing, always clear your session and start fresh. When a colleague can’t reproduce your results, context bias is often why. The tools we use carry bias from previous queries.
Appetite vs. Crawl Budget
Frost reframed crawl budget for the AI era. With LLMs, she said, it’s about appetite, not budget. Everything is efficiency-driven.
“If you’re hungry and something tastes good, you want more of it. You go diving deeper into the meal. That’s like the LLMs. You give that first little taste and it likes it, it’s going all in. That’s why the training bot comes. If it’s not a good taste, if it’s not structured, it doesn’t have an appetite for it.”
She tested depth by making things progressively harder for the bot. Her finding: after about three pages deep, the bot stops wanting to go further. It’s not a hard limit—it’s efficiency. RAG bots know users won’t wait forever for an answer, so they optimize for speed.
The implication: dense, single-page content beats spread-out depth. If you want comprehensive coverage of a topic, put it on one well-structured page rather than spreading it across a deep hierarchy.
Frost’s Key Findings
Here’s what Frost validated through her experiments:
- Efficiency beats format. The bot will only visit you once properly. Make that one visit count with structured, accessible information.
- Retrieval appetite matters. Make the first page dense with your best content. That’s where you point them, that’s where they’ll feed.
- Context bias overrides architecture. First impressions stick within a session. Start with structured data or the bot will work with the unstructured version it cached.
- Structure enables reasoning. Do not publish pages without structured data. The bot can find unstructured content, but it can’t reason with it.
- Session throttling exists. The deeper into a session, the slower and less capable the bot becomes. Optimize for the first few pages.
- Embed, don’t link (for now). For lightweight content, embedded structured data won over routing to endpoints. Frost noted this might change as content gets more complex.
- No-index is a danger zone. If it’s not indexed, it’s not real for training data. You can send a RAG bot there, but it can’t store the information for future use. It cannot answer for you.
My Takeaways
What I appreciated most about Frost’s talk was the methodology. She didn’t just tell us what works—she showed us how to figure it out ourselves. Set up test sites. Stream logs in real time. Run parallel tests. Fail, learn, iterate.
Her point about SEOs owning this space resonated: “There should not be a new role for AI. SEOs should be adopting this because we are the most suited to do the job.”
Here’s what I’m taking back to my practice:
- Set up canary pages. Create test pages specifically to observe AI crawler behavior. Understand how deep they go, what they fetch, how they process structure.
- Test with RAG first. You can force RAG bots to visit specific URLs, which makes controlled experimentation possible. Training bots are harder to study.
- Structure everything. This was the throughline of the entire conference. Madhavan said it enables grounding. Van Berkel said it builds knowledge graphs. Frost proved it enables reasoning. There’s no excuse to skip structured data anymore.
- Clear sessions when testing. Context bias is real. If you can’t reproduce a result, check whether cached information is influencing the response.
- Optimize for appetite, not just crawl budget. Make your best content dense, accessible, and structured on pages that load fast. The AI’s efficiency requirements align with good SEO fundamentals—but the stakes are higher.
Frost closed with a challenge: “Test it. Learn it. Own it. And share it, because we don’t know what we don’t know.”
She’s right. The only way through the stone world is experimentation. Time to start building.
