Disallow vs Noindex: What's the Difference and When to Use Each
Disallow stops crawling, noindex stops indexing. Learn when to use each, why combining them backfires, and how to fix "Indexed, though blocked by robots.txt".
Aubrey Yung is an SEO Manager and Schema Markup Consultant.
Aubrey is a Google Search Central Gold Product Expert and a certified Product Marketer. She focuses on SEO Content Strategy, International SEO, Entity SEO, SERP analysis and schema markup.

Observatory
Observations and commentary on SEO events, Google announcements, conferences, and what is happening across the SEO, AEO, and GEO universe.
October 13, 2026
My notes from Day 3 of Google Search Central Live Deep Dive Europe 2026: Gary Illyes' timing estimates for crawling, indexing and serving, and what they mean in practice.
October 13, 2026
My notes from Day 3 of Google Search Central Live Deep Dive Europe 2026: how Google detects language, drops stopwords, recognises entities, expands synonyms and fans out queries.
July 9, 2026
A first-hand look at Microsoft Clarity's new Topic Insights beta, how it tracks AI citation share, competitor visibility, and content gaps, plus its real limitations.
Blog
Disallow vs Noindex: What's the Difference and When to Use Each
Disallow stops crawling, noindex stops indexing. Learn when to use each, why combining them backfires, and how to fix "Indexed, though blocked by robots.txt".
What is @graph in schema markup?
Learn what @graph does in JSON-LD, when to use it instead of nesting, and how to connect entities with @id. Includes a full-page example and common mistakes.
Keyword Mapping: How to Map Keywords to Pages (With Examples)
Learn how to build a keyword map that matches every page to one search intent, avoids cannibalisation, and shows content gaps. Includes a free Google Sheets template.
What is a robots.txt file? How it works, with examples
Learn what robots.txt does (and doesn't do), how Google picks which rule applies, how to block AI crawlers, and the mistakes that get pages indexed anyway.
Service offering
I help companies solve complex SEO problems and build sustainable organic growth.
Get expert answers to specific SEO questions. A focused consultation to help you troubleshoot issues, validate ideas, or get guidance on complex SEO decisions.
A comprehensive analysis of your website’s SEO performance. I identify technical issues, content gaps, and growth opportunities with clear recommendations for improvement.
Build a long-term plan for sustainable organic growth. I work with your team to define priorities, align SEO with business goals, and guide execution over time.
Implement structured data that helps search engines better understand your content. I design and deploy schema markup to improve eligibility for rich results and enhance search visibility.
Identify and resolve technical issues that impact crawling, indexing, and performance. From site architecture to rendering and indexing, I help ensure search engines can properly access your site.
Optimize your website for multiple countries and languages. I help with hreflang implementation, international site structures, and localization strategies to reach global audiences.
Case Studies
Case studies show how strategy translated into measurable outcomes, clearer priorities, and execution that held up beyond the initial recommendation.
How an Information Architecture Overhaul Increased B2B SaaS Commercial Organic Traffic by 38%
Increasing trial signups by aligning intent
Driving +278% service-related keywords for a Hong Kong ophthalmology clinic