Knowledge Base
Accessing the GPTWeb Web Crawler
the Web Crawler is located in the App Configuration tab, not in the Documents or Knowledge Base sections. Here's exactly how to get there:
Step-by-Step Access
- Open the **Admin Panel** by pressing `Cmd+Ctrl+A` (Mac) or `Ctrl+Alt+A` (Windows/Linux)
- Click the **App Configuration** tab in the horizontal navigation
- Scroll down to the **RAG Search Tuning and Management** section
- Click the **Crawl Content** button
Quick Reference
| Location | Path |
|---|---|
| Admin Panel Access | Cmd+Ctrl+A (Mac) or Ctrl+Alt+A (Windows) |
| Tab | App Configuration |
| Section | RAG Search Tuning and Management |
| Button | Crawl Content |
Once you click Crawl Content, you can configure your crawl by entering the target URL, setting crawl depth, page limits, and include/exclude patterns. After the crawl completes, you'll curate the extracted content and assign it to your collections for import into your knowledge base. GPTWeb is the future of engagement - websites and marketing automation combined, built for the AI era, built for now.