# Website

> Read pages from your domain with a sitemap or a link crawl.

A **Website** source is one connector. You paste a URL. Datalumo checks that the host is reachable, looks for a sitemap, and then indexes pages on that domain.

WordPress, Shopify, and Laravel have their own [integrations](/docs/wordpress). If the check recognises those platforms, the wizard may suggest switching.

## Add a website

1. Open a knowledge base and add a **Website** source.
2. Paste the site URL (the homepage, or any page on the domain).
3. Run the check. Datalumo fetches the start page, probes `/sitemap.xml` and `/sitemap_index.xml`, and reads `Sitemap:` lines in `robots.txt`.
4. Pick how to discover pages, set a recrawl interval, and create the source.

Progress shows on the source page. The source flips to **Ready** when every page has been learned.

## Sitemap or crawl

After the check, you choose one discovery path. You can switch before you create the source.

| Path | When to use it |
|---|---|
| **Sitemap** | A sitemap was found, or you have a sitemap URL. Datalumo reads that file (and nested sitemap indexes) and indexes each page loc on the same registrable domain. |
| **Crawl** | No sitemap, or you want to follow links from the start page instead. Datalumo walks same-host links, honours `robots.txt`, and stops at your plan's page limit. |

Sitemap locs and nested sitemaps off your domain are dropped. The crawl stays on the same host. Tracking query parameters and asset URLs (images, scripts, stylesheets) are skipped.

## Recrawl

Choose how often Datalumo should fetch again:

- **Manual** (Check now only)
- **Daily**
- **Weekly** (default)
- **Monthly**

**Check now** runs the same path again at any time. Unchanged pages are skipped. Your plan's page allowance is the commercial cap on new pages.

## After it is Ready

Create a [chat](/docs/chat) or [search](/docs/search) widget on the same knowledge base, add your site domain, and [embed](/docs/sdk) it.

To keep a CMS in sync instead of crawling it, use [WordPress](/docs/wordpress), [Shopify](/docs/shopify), or [Laravel](/docs/laravel).