# Policy note (owner decision 2026-11-15): public pages are indexable again. # Crawling remains allowed; private pages retain their route-level noindex. # https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers # -------------------------------------------------- # Lumeitalia, Illuminazione LED Professionale # https://lumeitalia.it # Updated: 2026-11-15 | sito pubblico indicizzabile, aree private escluse # # Nota sui gruppi: un crawler segue SOLO il gruppo User-agent piu specifico che # lo nomina. I motori della sezione 1 hanno "Allow: /" e NON ereditano i # Disallow del gruppo "*": /ceo-dashboard è escluso esplicitamente; leggono # il meta robots noindex / 404 / 301 delle # aree interne e dei path legacy. I crawler AI (sezione 2) sono invece nel # gruppo "*" (sezione 5): sito pubblico aperto, aree private escluse. # -------------------------------------------------- # -------------------------------------------------- # 1) TRADITIONAL SEARCH ENGINE CRAWLERS # -------------------------------------------------- User-agent: Googlebot Allow: / Disallow: /ceo-dashboard User-agent: Googlebot-Image Allow: / Disallow: /ceo-dashboard User-agent: Googlebot-News Allow: / Disallow: /ceo-dashboard User-agent: Googlebot-Video Allow: / Disallow: /ceo-dashboard User-agent: AdsBot-Google Allow: / Disallow: /ceo-dashboard User-agent: Mediapartners-Google Allow: / Disallow: /ceo-dashboard User-agent: Storebot-Google Allow: / Disallow: /ceo-dashboard User-agent: Bingbot Allow: / Disallow: /ceo-dashboard User-agent: MicrosoftPreview Allow: / Disallow: /ceo-dashboard User-agent: BingPreview Allow: / Disallow: /ceo-dashboard User-agent: DuckDuckBot Allow: / Disallow: /ceo-dashboard User-agent: Twitterbot Allow: / Disallow: /ceo-dashboard User-agent: facebookexternalhit Allow: / Disallow: /ceo-dashboard User-agent: LinkedInBot Allow: / Disallow: /ceo-dashboard User-agent: Pinterest Allow: / Disallow: /ceo-dashboard User-agent: Slackbot Allow: / Disallow: /ceo-dashboard User-agent: WhatsApp Allow: / Disallow: /ceo-dashboard User-agent: TelegramBot Allow: / Disallow: /ceo-dashboard User-agent: Discordbot Allow: / Disallow: /ceo-dashboard User-agent: Yandex Allow: / Disallow: /ceo-dashboard User-agent: YandexImages Allow: / Disallow: /ceo-dashboard User-agent: Baiduspider Allow: / Disallow: /ceo-dashboard User-agent: Sogou Allow: / Disallow: /ceo-dashboard User-agent: Naverbot Allow: / Disallow: /ceo-dashboard User-agent: Yeti Allow: / Disallow: /ceo-dashboard User-agent: Applebot Allow: / Disallow: /ceo-dashboard User-agent: Qwantify Allow: / Disallow: /ceo-dashboard User-agent: Seznam Allow: / Disallow: /ceo-dashboard # -------------------------------------------------- # 2) AI ANSWER ENGINES & LLM CRAWLERS (2026) # Accesso aperto per la ricerca e la citazione in ChatGPT, Claude, Perplexity, # Copilot, Mistral, DuckDuckGo, Amazon... (bot di ricerca e agenti utente). # I crawler di addestramento (GPTBot, ClaudeBot, Google-Extended, CCBot...) # sono esclusi nella sezione 3. # I loro User-agent sono elencati nel gruppo "*" (sezione 5): stesso # accesso aperto al sito pubblico, ma fuori da /admin, /api, /auth, # /workspace e dagli URL con parametri di tracking. Con un gruppo proprio # con solo "Allow: /" non ereditavano nessuno di questi Disallow. # -------------------------------------------------- # -------------------------------------------------- # 3) BLOCKED CRAWLERS: AI / search crawlers excluded by policy # -------------------------------------------------- # ByteDance (Bytespider): bloccato User-agent: Bytespider Disallow: / # Huawei Petal Search (PetalBot): bloccato User-agent: PetalBot Disallow: / # Addestramento di modelli, dataset ed estrazione massiva: esclusi. # La ricerca AI e gli agenti che leggono una pagina su richiesta di una # persona restano ammessi (sezione 5). Diritti TDM riservati: /.well-known/tdmrep.json User-agent: GPTBot User-agent: anthropic-ai User-agent: Claude-Web User-agent: ClaudeBot User-agent: Google-Extended User-agent: Google-CloudVertexBot User-agent: GoogleOther User-agent: Applebot-Extended User-agent: FacebookBot User-agent: Meta-ExternalAgent User-agent: Amazonbot User-agent: cohere-ai User-agent: cohere-training-data-crawler User-agent: CCBot User-agent: Diffbot User-agent: AI2Bot User-agent: Ai2Bot-Dolma User-agent: PanguBot User-agent: img2dataset User-agent: Webzio-Extended User-agent: ImagesiftBot User-agent: omgili User-agent: omgilibot Disallow: / # Programmi di copia integrale del sito (mirroring offline): esclusi. User-agent: HTTrack User-agent: Wget User-agent: WebCopier User-agent: WebZIP User-agent: Offline Explorer User-agent: Teleport User-agent: TeleportPro User-agent: WebStripper User-agent: WebReaper User-agent: SiteSnagger User-agent: WebWhacker User-agent: Website Downloader User-agent: Website-Copier User-agent: SiteSucker User-agent: Scrapy Disallow: / # -------------------------------------------------- # 4) BLOCK SCRAPERS / SEO TOOLS / SPAM # -------------------------------------------------- User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: SerpstatBot Disallow: / User-agent: rogerbot Disallow: / User-agent: SiteAuditBot Disallow: / User-agent: ZoominfoBot Disallow: / User-agent: MegaIndex Disallow: / User-agent: BacklinkCrawler Disallow: / User-agent: Mail.RU_Bot Disallow: / User-agent: linkfluence Disallow: / User-agent: serpstatbot Disallow: / # -------------------------------------------------- # 5) GENERAL RULES (catch-all: crawler senza gruppo dedicato) # -------------------------------------------------- # OpenAI User-agent: ChatGPT-User User-agent: OAI-SearchBot # Anthropic User-agent: Claude-SearchBot User-agent: Claude-User # Perplexity User-agent: PerplexityBot User-agent: Perplexity-User # Meta / Facebook AI User-agent: Meta-ExternalFetcher # Amazon User-agent: Amzn-SearchBot User-agent: Amzn-User # DuckDuckGo AI User-agent: DuckAssistBot # You.com User-agent: YouBot # Mistral AI User-agent: Mistral-AI-User User-agent: MistralAI-User # Kagi User-agent: Kagibot # Brave Search AI User-agent: Bravebot # Timpi User-agent: Timpibot # AndiSearch User-agent: AndiBot # Phind / Copilot User-agent: PhindBot # xAI / Grok User-agent: xAI-Grok User-agent: GrokBot # Meta AI User-agent: Meta-AI # Cohere Coral User-agent: Cohere-Coral # Komo User-agent: KomoBot # Tutti gli altri crawler User-agent: * Allow: / Disallow: /ceo-dashboard # Private / admin / internal areas # I gruppi nominati dei motori di ricerca (Googlebot, Bingbot...) NON ereditano # queste regole: salvo /ceo-dashboard, restano Allow / cosi possono leggere il # meta robots noindex servito nell'HTML iniziale delle aree interne. Disallow: /admin/ Disallow: /workspace Disallow: /.lovable/ Disallow: /interno/ Disallow: /api/ Disallow: /dashboard/ Disallow: /auth Disallow: /_ Disallow: /operator/ Disallow: /iot/ Disallow: /gestione-relamping Disallow: /relamping-tigota Disallow: /tigota Disallow: /datasheet-generator Disallow: /generatore-codici Disallow: /xperience Disallow: /area-admin Disallow: /support-hub/ Disallow: /intelligence-hub Disallow: /evacuazione Disallow: /secret Disallow: /admin/jsonld-validator Disallow: /verifica-consenso Disallow: /unsubscribe Disallow: /verifica-email Disallow: /registration-success Disallow: /accesso-remoto # Legacy CMS / dead paths Disallow: /_dm/ Disallow: /1/ Disallow: /search Disallow: /pagina-vuota Disallow: /catalogo-2022 Disallow: /catalogo-esterno Disallow: /index.html Disallow: /index Disallow: /en-gb/ Disallow: /preventivo Disallow: /prenota Disallow: /area-riservata # Catalogo paginato, preserva canonical su /catalogo-sfogliabile Disallow: /catalogo-sfogliabile?page=* # Osservatorio, vecchio slug NDA (301 - /osservatorio/pavimenti-luminosi-dmx-uniqua-light-os) Disallow: /osservatorio/lumeitalia-illumina-sfilata-uniqua-light-giugno-2026 # Duplicate/draft content patterns Disallow: /copia-di- Disallow: /drawing- Disallow: /prova- Disallow: /home2023 Disallow: /Prodotti2026 Disallow: /xxx Disallow: /newpage # Legacy WIX product/page slugs (canonical lives under /prodotti/{slug}) # Rispondono 404 noindex o 301. Valgono solo per i crawler del gruppo "*": # Googlebot/Bingbot li leggono comunque (gruppo dedicato) e ne vedono noindex/301. Disallow: /michelangelo- Disallow: /evo-20- Disallow: /oldcity20 Disallow: /pad22 Disallow: /pad-street Disallow: /pad2283b38dcf Disallow: /pandora-tower Disallow: /pandora-tw Disallow: /pandora-b Disallow: /kit-retrofit Disallow: /alia-sport- Disallow: /alia-bay- Disallow: /alia-b-s Disallow: /matrix-fl- Disallow: /matrix-r- Disallow: /goaled20- Disallow: /newton20- Disallow: /italico-20- Disallow: /italico-22 Disallow: /europa-20- Disallow: /thor20-fl- Disallow: /thor20-ga- Disallow: /thor-lx-fl- Disallow: /wanda-linear Disallow: /urban-decorativo Disallow: /indoor-architectural Disallow: /indoor-industrial Disallow: /indoor-retail Disallow: /outdoor-architectural Disallow: /industrial811f15ef Disallow: /applicazioni-tab Disallow: /key-energy Disallow: /conosciamoci Disallow: /Azienda Disallow: /azienda-new Disallow: /street-lighting # /indoor - pagina pubblica (categoria indoor), non bloccata Disallow: /news$ Disallow: /podcast Disallow: /gallery Disallow: /contact # /lavora-con-noi - pagina pubblica indicizzabile (carriere), non bloccata Disallow: /mobi/ # Block tracking parameters (prevent duplicate content) # Parametri che generano duplicati infiniti (cache-buster del recovery chunk # e toggle di vista del catalogo). Nessuna risorsa JS/CSS/immagine è coinvolta. Disallow: /*?_cb= Disallow: /*?view= Disallow: /*?utm_* Disallow: /*&utm_* Disallow: /*?fbclid=* Disallow: /*&fbclid=* Disallow: /*?gclid=* Disallow: /*&gclid=* Disallow: /*?msclkid=* Disallow: /*&msclkid=* Disallow: /*?mc_cid=* Disallow: /*?mc_eid=* Disallow: /*?ref=* Disallow: /*?source=* Disallow: /*?remove_item=* Disallow: /*?_wpnonce=* Disallow: /*?trk=* Disallow: /*?lang=en* # Pagine indicizzabili che condividono il prefisso con un Disallow legacy # (/kit-retrofit, /preventivo): la regola Allow piu lunga prevale. Allow: /kit-retrofit-led$ Allow: /preventivo-illuminazione-led$ # Allow legitimate query patterns Allow: /prodotti?q=* Allow: /prodotti?category=* Allow: /shop?category=* Allow: /risorse?topic=* # Block non-existent language feeds Disallow: /es/blog-feed.xml Disallow: /ar/blog-feed.xml Disallow: /de/blog-feed.xml # Allow static assets explicitly Allow: /assets/ Allow: /img/ Allow: /og/ Allow: /images/ Allow: /downloads/ Allow: /cataloghi/ Allow: /docs/ Allow: /*.js$ Allow: /*.css$ Allow: /*.webp$ Allow: /*.avif$ Allow: /*.png$ Allow: /*.jpg$ Allow: /*.jpeg$ Allow: /*.svg$ Allow: /*.ico$ Allow: /*.woff2$ Allow: /*.pdf$ # Explicitly allow AI/LLM discovery files Allow: /llms.txt Allow: /llms-full.txt Allow: /ai.txt Allow: /humans.txt Allow: /.well-known/ # -------------------------------------------------- # 6) SITEMAPS (machine-readable index) # -------------------------------------------------- # Indici (sitemap.xml e sitemap-index.xml elencano le stesse sitemap figlie), # poi ogni sitemap figlia pubblicata in dist/. Sitemap: https://lumeitalia.it/sitemap.xml Sitemap: https://lumeitalia.it/sitemap-index.xml Sitemap: https://lumeitalia.it/sitemap-pages.xml Sitemap: https://lumeitalia.it/sitemap-applicazioni.xml Sitemap: https://lumeitalia.it/sitemap-prodotti.xml Sitemap: https://lumeitalia.it/sitemap-articoli.xml Sitemap: https://lumeitalia.it/sitemap-articoli-en.xml Sitemap: https://lumeitalia.it/sitemap-progetti.xml Sitemap: https://lumeitalia.it/sitemap-tecnologie.xml Sitemap: https://lumeitalia.it/sitemap-normative.xml Sitemap: https://lumeitalia.it/sitemap-guide.xml Sitemap: https://lumeitalia.it/sitemap-immagini.xml Sitemap: https://lumeitalia.it/sitemap-citta.xml Sitemap: https://lumeitalia.it/sitemap-domande.xml Sitemap: https://lumeitalia.it/sitemap-keyword.xml Sitemap: https://lumeitalia.it/sitemap-migliori.xml Sitemap: https://lumeitalia.it/sitemap-ai.xml Sitemap: https://lumeitalia.it/sitemap-osservatorio.xml Sitemap: https://lumeitalia.it/sitemap-magazine.xml Sitemap: https://lumeitalia.it/sitemap-magazine-en.xml Sitemap: https://lumeitalia.it/sitemap-live.xml Sitemap: https://lumeitalia.it/sitemap-i18n.xml # -------------------------------------------------- # 7) AI / LLM DISCOVERY (proposed standards) # -------------------------------------------------- # https://lumeitalia.it/llms.txt indice delle pagine canoniche per LLM (https://llmstxt.org) # https://lumeitalia.it/llms-full.txt riferimento completo: prodotti, specifiche, guide, FAQ # https://lumeitalia.it/ai.txt policy di citazione e uso AI # llms.txt e llms-full.txt sono generati in prebuild da scripts/generate-llms-txt.mjs.