Knowledge Base
Build the knowledge behind your agent on botts.ai. Crawl your website, upload documents, manage sources, run AI search, and understand page and storage limits. Then link a Knowledge Base to an agent for retrieval.
A Knowledge Base is the Information layer, the first of the Three I's. It is where you feed your agent the facts it needs: crawl your website or upload documents, and botts.ai indexes the content so your agent can retrieve accurate answers in real time.
Knowledge Bases are independent of agents. One agent can be linked to multiple Knowledge Bases, and one Knowledge Base can be shared by multiple agents. For example, a single product catalogue can feed both a support bot and a sales bot.
You manage Knowledge Bases under Knowledge in the left sidebar.
Who can do what
Browsing a Knowledge Base and creating or changing one require different roles.
| Action | Required role |
|---|---|
| View Knowledge Bases, browse sources and documents, view chunk text, run AI search | Builder, Admin, Owner |
| Create, rename, delete a Knowledge Base | Admin, Owner |
| Add or delete sources, upload files, trigger crawls, change crawl settings | Admin, Owner |
Builders can browse a Knowledge Base, open documents, and run AI search, but they cannot create, edit, delete, crawl, or upload: those write actions are Admin and Owner only. Members have no access to the Knowledge surface and are redirected to Chat. The server enforces these rules on every request. Builders still see the create, upload and crawl controls, but those actions return a permission error.
Creating a Knowledge Base
- Go to Knowledge in the sidebar.
- Click New Knowledge Base.
- Enter a Name (up to 255 characters) and click Create Knowledge Base.
Creating a Knowledge Base needs only a name: there is a single input on the form. You are taken to the new Knowledge Base's detail page, where you add sources.
You can rename a Knowledge Base later from the detail page header using the pencil icon (a renamed name is capped at 200 characters). Deleting a Knowledge Base is done from the trash icon on its card in the list, behind a confirmation dialog.
The Knowledge Base list
The list page shows a usage banner with two bars at the top, followed by a card for each Knowledge Base.
Each card shows the name, a short slug ID, and counts for sources, documents, and pages, plus total stored size in MB. Click a card to open its detail page; use the trash icon to delete it.
The two usage bars are org-wide:
- Knowledge-base pages: how much of your Knowledge Base size allowance you have used (used / limit pages, or
∞if unlimited). - Upload storage: the combined original size of your uploaded files in MB.
A bar turns red once usage passes 90%.
Adding sources
A Knowledge Base is built from sources. There are two kinds:
- Website: a URL that botts.ai crawls and indexes.
- File: a document you upload.
Add a website to crawl
In the Sources section, type a URL into the URL field (it shows https://example.com) and click Add. If you leave off http:// or https://, botts.ai prepends https:// automatically. Adding a website creates the source; you then start indexing it with the Play (crawl) button. If the address permanently redirects elsewhere (for example from example.ch to www.example.com), the crawl follows the redirect and indexes the site where it actually lives.
Upload documents
Drag files onto the upload area, or click it to pick files. You can select multiple files at once. Supported file types are:
| Type | Extensions |
|---|---|
.pdf | |
| Word | .doc, .docx |
| Text | .txt |
| Markdown | .md |
For reliable text extraction, save older .doc files as .docx or PDF before uploading.
Each file can be up to 100 MB. Larger files are rejected before upload.
File sources show a Download button that returns the original file. Website sources show Play (crawl) and a gear (settings) button instead. Every source has a Delete button.
Crawling a website
Crawling fetches pages from a website source, extracts their text, and indexes it. Start a crawl with the Play button on a website source. While a crawl runs, the interface polls for progress every few seconds and the source shows a live crawling (N pages) status that you can click to open the Crawl Progress modal.
A website source moves through these states:
| Status | Meaning |
|---|---|
| Not indexed | The source was added but never crawled. |
| Queued | Waiting for a free crawl slot. It starts automatically and shows queued…. |
| Crawling | A crawl is running; shows crawling (N pages). |
| Indexed (N) | The crawl finished and N documents are indexed. |
| No extractable text | The crawl finished but found no text to index (for example an image-only PDF). |
| Failed | The crawl failed; an error message is shown. |
| Incomplete (N pages) | The crawl was interrupted and auto-recovered after N pages. Try again. |
Crawl settings
Open the gear icon on a website source to configure how it is crawled.
| Setting | What it does |
|---|---|
| Crawl Schedule | How often to re-crawl: Manual (On demand), Daily, Weekly or Monthly. |
| Max Pages | The most pages to fetch in one crawl (1–10,000; default 100). |
| Max Depth | How many links deep to follow from the start URL (1–5; default 3). |
| Exclude subdomains | When on, stays on the main domain and skips subdomains. Off by default. |
| Include linked PDFs | When on, downloads and indexes PDFs linked from crawled pages. Off by default. |
| Skip these paths | URL paths to skip, one per line; * wildcards allowed. |
| Only crawl these paths | Only crawl matching URL paths, one per line; * wildcards allowed. Leave empty to crawl everything. |
| Skip pages containing these words | Pages containing any of these words are skipped, one per line. |
If the website publishes a sitemap, botts.ai also queues the pages it lists on the same domain, within your Max Pages and path filters. Max Depth only limits how far links are followed, so a low depth can still index many pages from a sitemap.
Manual re-crawls reuse the source's saved Max Pages and Max Depth. Changing the schedule to daily, weekly, or monthly registers an automatic recurring crawl; setting it back to manual (or deleting the source) removes it.
Max Pages is a per-crawl ceiling
Max Pages is a per-crawl ceiling, not your plan's storage limit. Crawls can run as long as you have credits. The size of your Knowledge Base is governed separately by the page limit described below. To keep a crawl within your remaining allowance, botts.ai automatically caps each crawl's extracted text to the headroom left under your plan's page limit.
Documents and full text
Indexed content appears in the Documents table, which you can sort (by title, source, type, status, pages or size, and date) and filter by title.
Document status reflects indexing:
| Status | Meaning |
|---|---|
| Processed | Text was extracted and embedded (green). |
| Processing | Still being indexed (yellow). |
| Skipped | No extractable text found on the page (orange). |
| Other | Any other state (gray). |
Each row shows a per-document pages count (one page is 2,000 characters of extracted text) and, for website documents, an external-link icon to the original URL. Click the eye icon to open the Document Viewer, which renders the document's full extracted text. The same full-text view is available from the Crawl Progress modal.
AI search and testing
The detail page includes built-in AI Search so you can test what your agent will retrieve before linking the Knowledge Base to it. In the Documents section header, pick a search mode, type a query into the AI Search content... box and click Search. The table is replaced by Search Results: up to 20 matching chunks, grouped by source, each with a percentage match. Click Clear to return to the documents.
Choose a search mode:
| Mode | Best for |
|---|---|
| Hybrid (recommended, default) | Balances semantic meaning, keyword matching, and document metadata. The best general choice. |
| Semantic | Conceptual or vaguely worded questions, where meaning matters more than exact words. |
| Keyword | Exact keyword matches such as names, product codes, or acronyms. |
A Reranker toggle (on by default) re-orders results so the most relevant chunks appear first. Results are grouped by source URL and show a percentage match; results that have no similarity score are labelled Keyword match. AI Search is available to Builders, Admins, and Owners.
Linking a Knowledge Base to an agent
A Knowledge Base only helps an agent once it is linked. In an agent's configuration, open the Knowledge Bases (optional) section and tick one or more Knowledge Bases. The section appears once your organization has at least one Knowledge Base.
Linking is all you need. As soon as at least one Knowledge Base is linked, the agent gains knowledge retrieval automatically. There is no separate tool to switch on. At query time the agent searches all of its linked Knowledge Bases and uses the most relevant results to answer.
Under the hood the agent uses a knowledge_search tool that searches every linked Knowledge Base (hybrid mode by default) and returns the strongest results across all of them. The model can also choose fulltext mode for exact terms or vector mode for conceptual questions. If no Knowledge Base is linked, the tool simply reports that none are available.
Two optional read-only tools, knowledge_browse (list Knowledge Bases, sources, and documents) and knowledge_read (read a specific document or chunk), can be enabled with the Advanced Knowledge Base Browsing toggle under Built-in Tools. The toggle appears once at least one Knowledge Base is selected, and the tools can only reach Knowledge Bases linked to that agent.
Limits
Two separate org-wide limits govern Knowledge Base size, both shown on the usage bars. A limit of ∞ (unlimited) is never blocked.
Page limit (Knowledge Base size)
Your page limit covers all stored extracted text: crawled pages and uploaded documents combined. One page equals 2,000 characters of text (roughly one dense A4 page).
| Plan | Max pages |
|---|---|
| Personal | 800 |
| Business | 4,000 |
| Business Pro | 12,000 |
| Enterprise | Unlimited |
Upload limit (uploaded file size)
Your upload limit is the combined original size of your uploaded files in MB. Crawled content stores no raw file, so it never counts toward this limit. Files attached in Chat count too: documents, data files (CSV, XLSX, JSON) and images all count toward this limit and show on the same Upload storage bar. The text extracted from documents attached in Chat also counts toward your page limit.
| Plan | Max upload size |
|---|---|
| Personal | 200 MB |
| Business | 1,000 MB |
| Business Pro | 2,500 MB |
| Enterprise | Unlimited |
Grandfathered Lite organizations keep their previous limits (800 pages, 200 MB). A workspace without an active plan is locked.
When a limit is reached
Crawls are billed per page and indexing (embedding) is billed per token, so new crawls and uploads are blocked when your credit balance reaches zero. When you try to upload or crawl:
- Out of credits: the action is blocked until you top up your credits.
- At your page limit: uploads and crawls are blocked once your Knowledge Base is already at its page cap.
- Over your upload limit: an upload is blocked when the new file would push your combined uploaded size over the MB limit. Because the file size is known up front, this check is exact.
- Scheduled crawls are skipped while you are out of credits or at your page limit, and resume on the next scheduled run once there is room again.
The page limit only blocks when you are already at the cap, so a single large upload can slightly overshoot the page count, bounded by your upload (MB) limit.
Data residency and privacy
Knowledge Base data is stored in Switzerland for every organization, regardless of plan. Uploaded files and crawled documents live in Swiss object storage (Infomaniak), the search index is in a Swiss database, and embedding and reranking run on Swiss infrastructure. To crawl a website source, botts.ai fetches and renders its public pages on hosted crawling infrastructure from Modal Labs (USA), which is listed among our subprocessors. The extracted text is then stored in Switzerland. Uploaded files are processed in Switzerland and never go to the crawler. botts.ai does not train AI models on your data.
Next steps
- Agents →: Link a Knowledge Base to an agent and write its system prompt.
- Quickstart →: Build your first Knowledge Base from a website and files.
Last updated on October 28, 2026